-- Logs begin at Tue 2022-10-04 11:07:20 UTC, end at Tue 2022-10-04 12:25:18 UTC. -- Oct 04 11:38:53.723339 dv-cinder-860212-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 04 11:38:53.732325 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 4 11:38:53 2022] *** Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: nodename: dv-cinder-860212-2-iscsi Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: machine: x86_64 Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: clock source: unix Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: pcre jit disabled Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: detected number of CPU cores: 4 Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: current working directory: / Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: detected binary path: /usr/bin/uwsgi-core Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: your processes number limit is 47680 Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: your memory page size is 4096 bytes Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: detected max file descriptor number: 2048 Oct 04 11:38:53.734929 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: lock engine: pthread robust mutexes Oct 04 11:38:53.735679 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: thunder lock: enabled Oct 04 11:38:53.735679 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 04 11:38:53.735679 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 04 11:38:53.774605 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: Python main interpreter initialized at 0x55727e5b89c0 Oct 04 11:38:53.774605 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: python threads support enabled Oct 04 11:38:53.774605 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: your server socket listen backlog is limited to 100 connections Oct 04 11:38:53.774605 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: your mercy for graceful operations on workers is 90 seconds Oct 04 11:38:53.774955 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: mapped 403077 bytes (393 KB) for 2 cores Oct 04 11:38:53.775032 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: *** Operational MODE: preforking *** Oct 04 11:38:53.775301 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: *** uWSGI is running in multiple interpreter mode *** Oct 04 11:38:53.775301 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: spawned uWSGI master process (pid: 93200) Oct 04 11:38:53.775510 dv-cinder-860212-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 04 11:38:53.778902 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: spawned uWSGI worker 1 (pid: 93212, cores: 1) Oct 04 11:38:53.778902 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: spawned uWSGI worker 2 (pid: 93213, cores: 1) Oct 04 11:38:53.778902 dv-cinder-860212-2-iscsi devstack@keystone.service[93200]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 04 11:38:58.168263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 11:38:58.168263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: from cryptography.utils import int_from_bytes Oct 04 11:38:58.169087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 11:38:58.169087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: from cryptography.utils import int_from_bytes Oct 04 11:38:58.171353 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 11:38:58.171353 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: from cryptography.utils import int_from_bytes Oct 04 11:38:58.171824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 11:38:58.171824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: from cryptography.utils import int_from_bytes Oct 04 11:38:58.228797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 11:38:58.228978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 11:38:58.229068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 11:38:58.229166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 11:38:58.229248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] command line args: [] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 11:38:58.229327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] command line args: [] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 11:38:58.229429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 11:38:58.229524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 11:38:58.229604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ================================================================================ {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 11:38:58.229682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ================================================================================ {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 11:38:58.229855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] admin_token = **** {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.229942 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] admin_token = **** {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.230098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] config_dir = [] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.230190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] config_dir = [] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.230294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.230381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.230466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] config_source = [] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.230543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] config_source = [] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.230653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] debug = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.230783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] debug = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.230966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.231172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.231266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.231340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.231482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.231837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.231949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.232047 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.232196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.232334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.232453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] list_limit = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.232558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] list_limit = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.232664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] log_config_append = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.232770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] log_config_append = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.232874 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.232986 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.233087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] log_dir = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.233191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] log_dir = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.233848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] log_file = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.234015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] log_file = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.234250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.234344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.234418 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.234538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.234538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.234731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.234808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.234933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235020 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.235966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.236089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.236200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.236277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.251861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.251967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] publish_errors = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] publish_errors = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.252995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253142 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253298 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] standard_threads = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] standard_threads = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.253921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] use_journal = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] use_journal = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] use_json = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254601 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] use_json = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] use_stderr = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] use_stderr = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] use_syslog = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] use_syslog = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.254979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.255103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 11:38:58.255186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.255266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.255339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.255409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.255491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.255564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.255639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.255711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.255787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.255864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.255946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256024 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256728 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.256948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.257996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258369 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258594 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.258959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259265 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.259942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.260048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.260158 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.260293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.260400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.260400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.260594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.260700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.260807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.260906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.261010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.261010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.261248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.261335 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.261422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.261535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.261658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.261741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.261860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.261936 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262709 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.262979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263181 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263532 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263728 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.263993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.264071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.264152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.264243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.264243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.264378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.264453 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265786 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.265942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266008 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266174 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.266928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.267981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.268994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269628 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.269930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.270971 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271117 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271694 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.271990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272776 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] auth.external = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] auth.external = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.272931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] auth.password = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] auth.password = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] auth.token = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] auth.token = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273862 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.273942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] credential.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] credential.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.274977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.275965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276111 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.276965 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] federation.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277346 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] federation.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.277934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278020 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.278994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279573 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.279933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.280009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.280088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.280207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.280807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.280888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.280961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281040 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281114 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.281971 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282030 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282177 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282321 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282655 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282728 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.282962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283109 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.283962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284539 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.284985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285131 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285273 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.285820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.286070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.286206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.286334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.286462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.286590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.286726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.286855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.286982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.287107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.287246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.287375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.287514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.287648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.287776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.287902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.287978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.288041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.288196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.288322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.288452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.288581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.288722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.288860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.288996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.289124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.289124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.289366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.289366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.289607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.289742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.289869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.289997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.290123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.290123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.290361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.290495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.290636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.290764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.290890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.291017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.291143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.291277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.291405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.291531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.291657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.291783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.291911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.292050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.292199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.292326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.292465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.292592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.292718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.292855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.292983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.293109 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.293236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.293319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.293438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.293570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.293736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.293819 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.293892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.293965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294038 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.294938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.295076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.295166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.295248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.295360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.295442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.295516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.295589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.295671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.295746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.295941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.296055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.296239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300253 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300410 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] resource.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300721 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] resource.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.300942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301172 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.301984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] role.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] role.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] role.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] role.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.302991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303178 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303346 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303567 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.303939 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304083 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.304966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305272 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.305995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.306951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.307098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.307174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.307324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.307402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.307547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] token.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.307633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.307777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.307855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.308064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] token.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.308141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.308286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.308523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.308523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.308775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.308906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.309033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.309033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.309282 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.309544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.309682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.309810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.309937 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.310074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.310243 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.310377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.310512 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.310641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.310774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.310911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.311039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.311164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.311294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.311909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.312047 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.312269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.312410 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.312541 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.312670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.312798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.312933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.connection = **** {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.313069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.313200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.313328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.313528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.313658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.connection = **** {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.313787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.313920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.314059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.314187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.314324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.314452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.314580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.314707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.314835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.314964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.315099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.315227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.315355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.315516 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.315648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.315776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.315924 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.316052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.316193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.316276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.316447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.316570 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.316709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.316846 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.316973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.317099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.317228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.317355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.317490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.317703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.317832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.317964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.318091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 11:38:58.318227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.318227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 11:38:58.318227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 11:38:58.346558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.346784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.346963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.347114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.347250 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.347394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.355545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.355803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.384450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.384673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.385847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.386081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.438377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.443608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.486794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.487018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.488559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.488797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.643322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.643595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.643734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.643877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.644014 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.644150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.644331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.644463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.644605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.644740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.644873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.644873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.667472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.667662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.667797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.667927 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.668070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.668233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.668376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.668376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.668638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.668769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.668908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.668993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.669168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.669168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.679803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.680372 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.693156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.693364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.693498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.693588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.693769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.693905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.694059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.694191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.694339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.694470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.694613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.694756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.694902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.694902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 11:38:58.711694 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.711896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 11:38:58.712259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.712482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 11:38:58.712771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.712853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.713023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 11:38:58.713106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.713279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.713362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.713534 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.713621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 11:38:58.713809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.714073 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.714337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.714595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.724161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.724418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.724504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.724701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.724785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.724977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.725061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.725263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.732274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.732641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.735671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.735952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.736213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.736302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.736506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.736589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.736791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.736893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.737060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.737144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.737331 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.737514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.737605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.737806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.737892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.738082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.738165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.738360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.738452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.738634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.738910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.739181 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.761646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.762516 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.763143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.763427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.763981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.764292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.768083 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.768389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.768669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.769124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.769414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.769700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.775144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.775427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.776050 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.776365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.781649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.781952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.782242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.782378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.782524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.782639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.782795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.782918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.783063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.783176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.783390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.783507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.783755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.783899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.784016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.784204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.800709 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.800873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.801035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.801179 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.804941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.805079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.808104 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.808356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.808492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.808655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.813163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.813465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.816214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.816687 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.819364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.819659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.819942 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.820072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.820232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.820381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.820522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.820651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.820779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.820907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.821107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.821398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.836391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.836679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.836944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.837209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.837351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.837618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.837887 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.838151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.844077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.845106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.845622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.846641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.847146 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.847420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.848294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.848586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.850077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.850361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.850644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.850924 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.851283 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.851422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.851554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.851841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.852126 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.852536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.863314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.863598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.863877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.864156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.864554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.864694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.864832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.864961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.865090 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.865217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.865401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.865782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.866065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.866348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.881041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.882329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.883906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.885244 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.886785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.888160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.888381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.889623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.889764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.890003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.890405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.891278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.891564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.891963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.896743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.898191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.898735 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.900337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.901136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.901531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.901816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.902091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.903224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.903621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.903909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.904209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.909933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.910232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.910514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.912206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.912507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.912791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.917438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.917754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.918007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.918281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.918554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.918824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.919098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.919383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.919651 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.919779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 11:38:58.919913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.920046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.920296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.920424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.920558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.920690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.921329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.921538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.921669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.921797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.921926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.922150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.922428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.922811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.923104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.923376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.923654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.923925 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 11:38:58.956110 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55727e5b89c0 pid: 93213 (default app) Oct 04 11:38:58.958526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55727e5b89c0 pid: 93212 (default app) Oct 04 11:38:58.959363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bdb5b-f6aa-4394-ae98-6e60e9eadf16 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:38:58.959651 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bdb5b-f6aa-4394-ae98-6e60e9eadf16 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:38:58.959922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bdb5b-f6aa-4394-ae98-6e60e9eadf16 None None] PATH_INFO: `/v3` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:38:58.964081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Tue Oct 4 11:38:58 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:07.162995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31139417-680f-47be-ad43-59754e2cd639 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:07.163520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31139417-680f-47be-ad43-59754e2cd639 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:07.163603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31139417-680f-47be-ad43-59754e2cd639 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:07.167871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:07.173068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:07.173551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:07.173809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:07.245858 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: INFO dbcounter [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] Registered counter for database keystone Oct 04 11:39:07.267400 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG oslo_db.sqlalchemy.engines [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93213) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 11:39:07.273715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writer thread running {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 11:39:07.457796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG passlib.handlers.bcrypt [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93213) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 11:39:07.458091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG passlib.handlers.bcrypt [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93213) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 11:39:07.516666 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:07.518283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:07.646707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 11:39:07.948819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31139417-680f-47be-ad43-59754e2cd639 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:08.019389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: INFO dbcounter [None req-31139417-680f-47be-ad43-59754e2cd639 None None] Registered counter for database keystone Oct 04 11:39:08.038430 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG oslo_db.sqlalchemy.engines [None req-31139417-680f-47be-ad43-59754e2cd639 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93212) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 11:39:08.044705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writer thread running {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 11:39:08.164389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31139417-680f-47be-ad43-59754e2cd639 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:08.168818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e9f1a-86b9-4dc7-80dc-b62d40e61bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:08.169441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9f1a-86b9-4dc7-80dc-b62d40e61bab admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:08.169726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9f1a-86b9-4dc7-80dc-b62d40e61bab admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:08.170004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9f1a-86b9-4dc7-80dc-b62d40e61bab admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:08.178332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e9f1a-86b9-4dc7-80dc-b62d40e61bab admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:08.230384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a42e9f1a-86b9-4dc7-80dc-b62d40e61bab admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 11:39:08.230734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e9f1a-86b9-4dc7-80dc-b62d40e61bab admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:08.240840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Tue Oct 4 11:39:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 11:39:08.246799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:08.265000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b35bf1-9ab0-4a73-9ca5-e12e7a3ef087 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:08.269459 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a780d-2fc1-48be-a3ab-012abe96cd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:08.270567 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a780d-2fc1-48be-a3ab-012abe96cd73 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:08.270852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a780d-2fc1-48be-a3ab-012abe96cd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:08.271133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a780d-2fc1-48be-a3ab-012abe96cd73 admin admin] PATH_INFO: `/v3/projects/a155f451dd324796b4bda64d212b8b97` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:08.272762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a780d-2fc1-48be-a3ab-012abe96cd73 admin admin] RBAC: Authorizing `identity:get_project(project_id=a155f451dd324796b4bda64d212b8b97)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:08.325094 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WARNING keystone.common.rbac_enforcer.enforcer [None req-521a780d-2fc1-48be-a3ab-012abe96cd73 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 11:39:08.325465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a780d-2fc1-48be-a3ab-012abe96cd73 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:08.330596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Tue Oct 4 11:39:08 2022] GET /identity/v3/projects/a155f451dd324796b4bda64d212b8b97 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:10.436569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a8f43-4863-4349-b7e2-e3f6f2750568 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:10.437106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a8f43-4863-4349-b7e2-e3f6f2750568 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:10.437106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a8f43-4863-4349-b7e2-e3f6f2750568 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:10.441159 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:10.446496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608945c-065d-4163-92a3-3b430a4fcf44 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:10.446910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608945c-065d-4163-92a3-3b430a4fcf44 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:10.447058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608945c-065d-4163-92a3-3b430a4fcf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:10.466977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a608945c-065d-4163-92a3-3b430a4fcf44 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:10.472964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a608945c-065d-4163-92a3-3b430a4fcf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:10.481107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 11:39:10.750631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a8f43-4863-4349-b7e2-e3f6f2750568 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:10.762539 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a8f43-4863-4349-b7e2-e3f6f2750568 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:10.767197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efb979-d0bf-4a33-9c65-504c1a763714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:10.767788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efb979-d0bf-4a33-9c65-504c1a763714 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:10.768068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efb979-d0bf-4a33-9c65-504c1a763714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:10.768377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efb979-d0bf-4a33-9c65-504c1a763714 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:10.775892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efb979-d0bf-4a33-9c65-504c1a763714 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:10.777691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efb979-d0bf-4a33-9c65-504c1a763714 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:10.787027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Tue Oct 4 11:39:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 11:39:10.792759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608945c-065d-4163-92a3-3b430a4fcf44 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:10.802006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608945c-065d-4163-92a3-3b430a4fcf44 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:10.806357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e4298-ec89-4b0e-ae42-ed16f0bdf04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:10.806983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e4298-ec89-4b0e-ae42-ed16f0bdf04e admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:10.807272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e4298-ec89-4b0e-ae42-ed16f0bdf04e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:10.807646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e4298-ec89-4b0e-ae42-ed16f0bdf04e admin admin] PATH_INFO: `/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:10.808992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e4298-ec89-4b0e-ae42-ed16f0bdf04e admin admin] RBAC: Authorizing `identity:get_user(user_id=374ef1a08ea74e7cb0bf480e25c8cdd4)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:10.810824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e4298-ec89-4b0e-ae42-ed16f0bdf04e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:10.812404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Tue Oct 4 11:39:10 2022] GET /identity/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:13.071086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c9cbd-d89c-47cc-8a4e-05107766801b None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:13.071581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c9cbd-d89c-47cc-8a4e-05107766801b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:13.071581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c9cbd-d89c-47cc-8a4e-05107766801b None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:13.072926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:39:13.077107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344b1b-b049-4e87-9fae-96a64f7d7114 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:13.077521 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344b1b-b049-4e87-9fae-96a64f7d7114 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:13.077662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344b1b-b049-4e87-9fae-96a64f7d7114 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:13.078187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cba32e-ca91-402c-aecc-9f576d85bcba None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:13.078476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cba32e-ca91-402c-aecc-9f576d85bcba None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:13.078758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cba32e-ca91-402c-aecc-9f576d85bcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:13.078963 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:13.083914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c05b1-f283-4ba6-a0d7-fdfcdcf64276 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:13.084238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c05b1-f283-4ba6-a0d7-fdfcdcf64276 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:13.084523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c05b1-f283-4ba6-a0d7-fdfcdcf64276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:13.102333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-813c05b1-f283-4ba6-a0d7-fdfcdcf64276 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:13.103524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-813c05b1-f283-4ba6-a0d7-fdfcdcf64276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:13.120923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG passlib.handlers.bcrypt [None req-35cba32e-ca91-402c-aecc-9f576d85bcba None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93212) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 11:39:13.121312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG passlib.handlers.bcrypt [None req-35cba32e-ca91-402c-aecc-9f576d85bcba None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93212) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 11:39:13.170956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-35cba32e-ca91-402c-aecc-9f576d85bcba None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:13.172608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-35cba32e-ca91-402c-aecc-9f576d85bcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:13.189483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 7/12] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:39:13.273585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 6/13] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:39:13.456261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c05b1-f283-4ba6-a0d7-fdfcdcf64276 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:13.479987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:13.538784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cba32e-ca91-402c-aecc-9f576d85bcba None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:13.562316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c05b1-f283-4ba6-a0d7-fdfcdcf64276 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:13.597632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2f080-7d8b-44d1-8256-e13ee70fbd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:13.598234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2f080-7d8b-44d1-8256-e13ee70fbd81 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:13.598531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2f080-7d8b-44d1-8256-e13ee70fbd81 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:13.598771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2f080-7d8b-44d1-8256-e13ee70fbd81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:13.642614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cba32e-ca91-402c-aecc-9f576d85bcba None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:13.645132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:13.670973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2f080-7d8b-44d1-8256-e13ee70fbd81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:13.672575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2f080-7d8b-44d1-8256-e13ee70fbd81 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:13.680699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4630a9-70b3-43e1-a58b-5f887752a444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:13.681811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4630a9-70b3-43e1-a58b-5f887752a444 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:13.682097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4630a9-70b3-43e1-a58b-5f887752a444 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:13.682371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4630a9-70b3-43e1-a58b-5f887752a444 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:13.694960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4630a9-70b3-43e1-a58b-5f887752a444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:13.696282 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-1b4630a9-70b3-43e1-a58b-5f887752a444 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:13.704373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:13.705318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:13.706078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:39:13.711531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4630a9-70b3-43e1-a58b-5f887752a444 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:13.772491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 8/15] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:13.778106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2f080-7d8b-44d1-8256-e13ee70fbd81 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:13.812807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4630a9-70b3-43e1-a58b-5f887752a444 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:13.845161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e44d4-bf7f-4ff3-93de-0fb6b245a5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:13.845759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e44d4-bf7f-4ff3-93de-0fb6b245a5ea None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:13.846042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e44d4-bf7f-4ff3-93de-0fb6b245a5ea None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:13.846316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e44d4-bf7f-4ff3-93de-0fb6b245a5ea None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:13.846971 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e44d4-bf7f-4ff3-93de-0fb6b245a5ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:13.847269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e44d4-bf7f-4ff3-93de-0fb6b245a5ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:13.848790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e44d4-bf7f-4ff3-93de-0fb6b245a5ea None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:13.861143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 8/16] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:13.866847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e44d4-bf7f-4ff3-93de-0fb6b245a5ea None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:13.877294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2f080-7d8b-44d1-8256-e13ee70fbd81 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:13.893701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:13.900472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:13.911722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219a3e6-7f2b-4ce5-be34-e9436ab9b81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:13.913678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219a3e6-7f2b-4ce5-be34-e9436ab9b81c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:13.913678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219a3e6-7f2b-4ce5-be34-e9436ab9b81c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:13.913678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219a3e6-7f2b-4ce5-be34-e9436ab9b81c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:13.923717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219a3e6-7f2b-4ce5-be34-e9436ab9b81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:13.925309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219a3e6-7f2b-4ce5-be34-e9436ab9b81c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-e219a3e6-7f2b-4ce5-be34-e9436ab9b81c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:39:13.943935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:13.948505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:39:13.949318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:39:13.949318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:39:13.949318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:13.949318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:39:13.952488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219a3e6-7f2b-4ce5-be34-e9436ab9b81c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:13.965671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e44d4-bf7f-4ff3-93de-0fb6b245a5ea None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:14.003359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c212fb-e91c-4318-bc32-dfdf0568b374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:14.003994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c212fb-e91c-4318-bc32-dfdf0568b374 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:14.004303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c212fb-e91c-4318-bc32-dfdf0568b374 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:14.004579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c212fb-e91c-4318-bc32-dfdf0568b374 None admin] PATH_INFO: `/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:14.006153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c212fb-e91c-4318-bc32-dfdf0568b374 None admin] RBAC: Authorizing `identity:get_user(user_id=374ef1a08ea74e7cb0bf480e25c8cdd4)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:14.007603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c212fb-e91c-4318-bc32-dfdf0568b374 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:14.009245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 9/18] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:13 2022] GET /identity/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:14.014830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c212fb-e91c-4318-bc32-dfdf0568b374 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:14.056310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219a3e6-7f2b-4ce5-be34-e9436ab9b81c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:14.093678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcce85a-0b0b-4533-a9d3-70b26f2c7348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:14.094351 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcce85a-0b0b-4533-a9d3-70b26f2c7348 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:14.094594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcce85a-0b0b-4533-a9d3-70b26f2c7348 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:14.094837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcce85a-0b0b-4533-a9d3-70b26f2c7348 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:14.095482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcce85a-0b0b-4533-a9d3-70b26f2c7348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:14.095793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcce85a-0b0b-4533-a9d3-70b26f2c7348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:14.097234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcce85a-0b0b-4533-a9d3-70b26f2c7348 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:14.108811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:14.115784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c212fb-e91c-4318-bc32-dfdf0568b374 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:14.116013 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 10/19] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:14.154408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d5abc-c1cf-4f7e-a52c-9ca185295bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:14.155602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d5abc-c1cf-4f7e-a52c-9ca185295bbf None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:14.155881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d5abc-c1cf-4f7e-a52c-9ca185295bbf None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:14.156149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d5abc-c1cf-4f7e-a52c-9ca185295bbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:14.157413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d5abc-c1cf-4f7e-a52c-9ca185295bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:14.158891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d5abc-c1cf-4f7e-a52c-9ca185295bbf None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:14.160514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 10/20] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:14.166555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcce85a-0b0b-4533-a9d3-70b26f2c7348 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:14.208634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:14.285435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcce85a-0b0b-4533-a9d3-70b26f2c7348 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:14.322391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdf463-c095-4b5f-adfe-0860ad88bca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:14.323024 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdf463-c095-4b5f-adfe-0860ad88bca7 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:14.323289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdf463-c095-4b5f-adfe-0860ad88bca7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:14.323567 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdf463-c095-4b5f-adfe-0860ad88bca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:14.324442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdf463-c095-4b5f-adfe-0860ad88bca7 None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.domain.id=default, user.id=374ef1a08ea74e7cb0bf480e25c8cdd4) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:14.324749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdf463-c095-4b5f-adfe-0860ad88bca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:14.326200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdf463-c095-4b5f-adfe-0860ad88bca7 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:14.333234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 11/21] 10.222.0.15 () {56 vars in 1322 bytes} [Tue Oct 4 11:39:14 2022] GET /identity/v3/role_assignments?user.id=374ef1a08ea74e7cb0bf480e25c8cdd4&scope.domain.id=default&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 233 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:16.534642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23835271-72b7-4520-adc6-8c2c4d41f9f8 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:16.535684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23835271-72b7-4520-adc6-8c2c4d41f9f8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:16.536238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23835271-72b7-4520-adc6-8c2c4d41f9f8 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:16.537781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 11/22] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:16.549827 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8628148-b563-4499-bc8f-dbb89d101b28 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:16.550377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8628148-b563-4499-bc8f-dbb89d101b28 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:16.550377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8628148-b563-4499-bc8f-dbb89d101b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:16.575925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d8628148-b563-4499-bc8f-dbb89d101b28 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:16.577265 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d8628148-b563-4499-bc8f-dbb89d101b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:16.667020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 12/23] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:16.998576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1da2b2-0ecc-4e62-a21b-2b175feace94 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:16.999263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1da2b2-0ecc-4e62-a21b-2b175feace94 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:16.999768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1da2b2-0ecc-4e62-a21b-2b175feace94 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:17.001417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 12/24] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:17.007026 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2c371-6d8a-4c73-a125-b11bf0051527 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:17.007315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2c371-6d8a-4c73-a125-b11bf0051527 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:17.007597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2c371-6d8a-4c73-a125-b11bf0051527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:17.037280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-5ba2c371-6d8a-4c73-a125-b11bf0051527 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:17.038543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5ba2c371-6d8a-4c73-a125-b11bf0051527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:17.071069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:17.126130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 13/25] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:17.234698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1da2b2-0ecc-4e62-a21b-2b175feace94 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:17.367829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1da2b2-0ecc-4e62-a21b-2b175feace94 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:17.415909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8a0c0-8188-4689-bcb9-730f9ea4cb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:17.416918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8a0c0-8188-4689-bcb9-730f9ea4cb49 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:17.417434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8a0c0-8188-4689-bcb9-730f9ea4cb49 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:17.417900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8a0c0-8188-4689-bcb9-730f9ea4cb49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:17.442543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8a0c0-8188-4689-bcb9-730f9ea4cb49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:17.444475 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8a0c0-8188-4689-bcb9-730f9ea4cb49 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-f6b8a0c0-8188-4689-bcb9-730f9ea4cb49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:39:17.465785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:17.466743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:39:17.467590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:39:17.467590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:39:17.467590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:17.467590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 13/26] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:17.468913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6617aff-fb15-4f6c-8c52-17f0f54a00b5 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:17.469205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6617aff-fb15-4f6c-8c52-17f0f54a00b5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:17.469479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6617aff-fb15-4f6c-8c52-17f0f54a00b5 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:17.470827 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 14/27] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:17.473206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8a0c0-8188-4689-bcb9-730f9ea4cb49 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:17.477970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690a862-e63b-429c-aaa8-e3f7e5e0effd None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:17.478294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690a862-e63b-429c-aaa8-e3f7e5e0effd None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:17.478574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690a862-e63b-429c-aaa8-e3f7e5e0effd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:17.493207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:17.525168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-3690a862-e63b-429c-aaa8-e3f7e5e0effd None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:17.526396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3690a862-e63b-429c-aaa8-e3f7e5e0effd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:17.614851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8a0c0-8188-4689-bcb9-730f9ea4cb49 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:17.670496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 15/28] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:39:17.671469 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690a862-e63b-429c-aaa8-e3f7e5e0effd None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:17.677527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08475019-b641-42a8-843d-e8749d593b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:17.678146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475019-b641-42a8-843d-e8749d593b9b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:17.678433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475019-b641-42a8-843d-e8749d593b9b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:17.678836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475019-b641-42a8-843d-e8749d593b9b None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:17.679492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08475019-b641-42a8-843d-e8749d593b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:17.679803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08475019-b641-42a8-843d-e8749d593b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:17.686980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08475019-b641-42a8-843d-e8749d593b9b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:17.728030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 14/29] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:17.735398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08475019-b641-42a8-843d-e8749d593b9b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:17.833202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:17.866337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08475019-b641-42a8-843d-e8749d593b9b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:17.880373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:17.905853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d74ac1-8365-45b7-88e5-8ab5cebdab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:17.906839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d74ac1-8365-45b7-88e5-8ab5cebdab13 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:17.907324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d74ac1-8365-45b7-88e5-8ab5cebdab13 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:17.907797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d74ac1-8365-45b7-88e5-8ab5cebdab13 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:17.908733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d74ac1-8365-45b7-88e5-8ab5cebdab13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:17.911314 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d74ac1-8365-45b7-88e5-8ab5cebdab13 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:17.914380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690a862-e63b-429c-aaa8-e3f7e5e0effd None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:17.962553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b09334-c930-420e-a5ff-93b15131e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:17.963171 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b09334-c930-420e-a5ff-93b15131e542 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:17.963457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b09334-c930-420e-a5ff-93b15131e542 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:17.963733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b09334-c930-420e-a5ff-93b15131e542 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:17.976754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b09334-c930-420e-a5ff-93b15131e542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-38b09334-c930-420e-a5ff-93b15131e542 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:17.983892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:17.984814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:17.985578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 16/30] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:17.987005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed850f8d-434d-4371-a5cd-ac8a482021a9 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:17.987005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed850f8d-434d-4371-a5cd-ac8a482021a9 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:17.987005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed850f8d-434d-4371-a5cd-ac8a482021a9 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:17.988547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 17/31] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:17.991470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed850f8d-434d-4371-a5cd-ac8a482021a9 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:18.108623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed850f8d-434d-4371-a5cd-ac8a482021a9 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:18.145760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef8959-3c29-4e40-a08c-d99d9c924501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:18.146399 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8959-3c29-4e40-a08c-d99d9c924501 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:18.146687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8959-3c29-4e40-a08c-d99d9c924501 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:18.146964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8959-3c29-4e40-a08c-d99d9c924501 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:18.147770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef8959-3c29-4e40-a08c-d99d9c924501 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:18.149330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef8959-3c29-4e40-a08c-d99d9c924501 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:18.168713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 15/32] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:39:17 2022] POST /identity/v3/projects => generated 301 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:39:18.169707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d74ac1-8365-45b7-88e5-8ab5cebdab13 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:18.369813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d74ac1-8365-45b7-88e5-8ab5cebdab13 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:18.377248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 18/33] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 4 11:39:17 2022] POST /identity/v3/roles => generated 225 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:39:18.378816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb334a40-e8f0-423f-b382-41f1b936d4aa None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:18.379107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb334a40-e8f0-423f-b382-41f1b936d4aa None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:18.379387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb334a40-e8f0-423f-b382-41f1b936d4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:18.427293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-eb334a40-e8f0-423f-b382-41f1b936d4aa None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:18.428935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-eb334a40-e8f0-423f-b382-41f1b936d4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:18.432522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:18.433172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:18.454731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ef2cf-74a5-4558-80b4-37231eb6f54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:18.455364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ef2cf-74a5-4558-80b4-37231eb6f54b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:18.455656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ef2cf-74a5-4558-80b4-37231eb6f54b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:18.455921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ef2cf-74a5-4558-80b4-37231eb6f54b None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:18.456641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ef2cf-74a5-4558-80b4-37231eb6f54b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:18.456964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ef2cf-74a5-4558-80b4-37231eb6f54b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:18.458738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ef2cf-74a5-4558-80b4-37231eb6f54b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:18.469600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 16/34] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:18.475700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ef2cf-74a5-4558-80b4-37231eb6f54b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:18.590561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 19/35] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:18.592595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0ddf5-057f-478d-bd21-bd8e88aa35f3 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:18.592885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0ddf5-057f-478d-bd21-bd8e88aa35f3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:18.593164 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0ddf5-057f-478d-bd21-bd8e88aa35f3 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:18.594640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 20/36] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:18.603585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353e222-4169-4625-a635-e537ef527b97 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:18.603585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353e222-4169-4625-a635-e537ef527b97 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:18.603585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353e222-4169-4625-a635-e537ef527b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:18.635974 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e353e222-4169-4625-a635-e537ef527b97 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:18.637306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e353e222-4169-4625-a635-e537ef527b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:18.643878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ef2cf-74a5-4558-80b4-37231eb6f54b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:18.684924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64327b2-5f68-48ae-bf5f-7a6db2470563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:18.686081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64327b2-5f68-48ae-bf5f-7a6db2470563 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:18.686388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64327b2-5f68-48ae-bf5f-7a6db2470563 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:18.686643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64327b2-5f68-48ae-bf5f-7a6db2470563 None admin] PATH_INFO: `/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:18.688125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64327b2-5f68-48ae-bf5f-7a6db2470563 None admin] RBAC: Authorizing `identity:get_user(user_id=374ef1a08ea74e7cb0bf480e25c8cdd4)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:18.689744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64327b2-5f68-48ae-bf5f-7a6db2470563 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:18.691386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 17/37] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:18 2022] GET /identity/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4 => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:18.693648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a572b-ea09-43ca-8284-f76374318428 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:18.693937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a572b-ea09-43ca-8284-f76374318428 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:18.694213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a572b-ea09-43ca-8284-f76374318428 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:18.696093 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 18/38] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:18.701161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a572b-ea09-43ca-8284-f76374318428 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:18.739507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 21/39] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:18.741468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0763da9-b554-453f-8e76-e3fb1013bf61 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:18.741763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0763da9-b554-453f-8e76-e3fb1013bf61 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:18.742048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0763da9-b554-453f-8e76-e3fb1013bf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:18.769913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a0763da9-b554-453f-8e76-e3fb1013bf61 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:18.770509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:18.771138 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a0763da9-b554-453f-8e76-e3fb1013bf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:18.825374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a572b-ea09-43ca-8284-f76374318428 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:18.858726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:18.866546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 22/40] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:18.875977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dec65-0f99-45ca-b3fd-ec1efdfdf2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:18.876944 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dec65-0f99-45ca-b3fd-ec1efdfdf2ea None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:18.877433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dec65-0f99-45ca-b3fd-ec1efdfdf2ea None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:18.877908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dec65-0f99-45ca-b3fd-ec1efdfdf2ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:18.879349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754dec65-0f99-45ca-b3fd-ec1efdfdf2ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:18.881053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754dec65-0f99-45ca-b3fd-ec1efdfdf2ea None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:18.882979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 19/41] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:18.889281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0763da9-b554-453f-8e76-e3fb1013bf61 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:18.889427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dec65-0f99-45ca-b3fd-ec1efdfdf2ea None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:19.013965 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dec65-0f99-45ca-b3fd-ec1efdfdf2ea None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:19.044322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:19.056058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2fd10-5bcb-4f2d-8584-13f29b08a961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:19.056795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2fd10-5bcb-4f2d-8584-13f29b08a961 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:19.057117 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2fd10-5bcb-4f2d-8584-13f29b08a961 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:19.057401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2fd10-5bcb-4f2d-8584-13f29b08a961 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:19.058084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2fd10-5bcb-4f2d-8584-13f29b08a961 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:19.059527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2fd10-5bcb-4f2d-8584-13f29b08a961 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:19.125057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0763da9-b554-453f-8e76-e3fb1013bf61 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:19.146492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 20/42] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 4 11:39:18 2022] POST /identity/v3/roles => generated 219 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:39:19.147255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2fd10-5bcb-4f2d-8584-13f29b08a961 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:19.173520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7febc-8603-4f34-b6d3-760882b19bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:19.174150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7febc-8603-4f34-b6d3-760882b19bf7 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:19.174481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7febc-8603-4f34-b6d3-760882b19bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:19.174773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7febc-8603-4f34-b6d3-760882b19bf7 None admin] PATH_INFO: `/v3/domains/default/users/374ef1a08ea74e7cb0bf480e25c8cdd4/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:19.176940 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7febc-8603-4f34-b6d3-760882b19bf7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=374ef1a08ea74e7cb0bf480e25c8cdd4, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:19.178583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7febc-8603-4f34-b6d3-760882b19bf7 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:19.259602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2fd10-5bcb-4f2d-8584-13f29b08a961 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:19.300060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96047575-aa29-4103-92ef-ae8eea54acd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:19.312583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 23/43] 10.222.0.15 () {58 vars in 1361 bytes} [Tue Oct 4 11:39:18 2022] PUT /identity/v3/domains/default/users/374ef1a08ea74e7cb0bf480e25c8cdd4/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:39:19.312818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96047575-aa29-4103-92ef-ae8eea54acd8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:19.312818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96047575-aa29-4103-92ef-ae8eea54acd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:19.312818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96047575-aa29-4103-92ef-ae8eea54acd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:19.313001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7febc-8603-4f34-b6d3-760882b19bf7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:19.313498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96047575-aa29-4103-92ef-ae8eea54acd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:19.315153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96047575-aa29-4103-92ef-ae8eea54acd8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:19.323765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 21/44] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:19.324847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96047575-aa29-4103-92ef-ae8eea54acd8 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:19.402171 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:19.416018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:19.460872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96047575-aa29-4103-92ef-ae8eea54acd8 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:19.469381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7febc-8603-4f34-b6d3-760882b19bf7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:19.504444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd3765-3e06-4a37-97e9-98cbb0ce24e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:19.505074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd3765-3e06-4a37-97e9-98cbb0ce24e5 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:19.505363 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd3765-3e06-4a37-97e9-98cbb0ce24e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:19.505649 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd3765-3e06-4a37-97e9-98cbb0ce24e5 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:19.506325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd3765-3e06-4a37-97e9-98cbb0ce24e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:19.507859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd3765-3e06-4a37-97e9-98cbb0ce24e5 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:19.510627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49a970-d356-487f-80a0-787d5f29fb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:19.511254 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a970-d356-487f-80a0-787d5f29fb21 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:19.513106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a970-d356-487f-80a0-787d5f29fb21 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:19.513106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a970-d356-487f-80a0-787d5f29fb21 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:19.513106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49a970-d356-487f-80a0-787d5f29fb21 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:19.514014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49a970-d356-487f-80a0-787d5f29fb21 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:19.604313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 24/45] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 4 11:39:19 2022] POST /identity/v3/roles => generated 223 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:39:19.632230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 22/46] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:39:19 2022] POST /identity/v3/projects => generated 312 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:39:21.528259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f66d1-8d1f-492e-92de-dd718cc20ee1 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:21.528828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f66d1-8d1f-492e-92de-dd718cc20ee1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:21.528828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f66d1-8d1f-492e-92de-dd718cc20ee1 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:21.530798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 25/47] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:21.536378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09181896-02d3-44e4-9497-78f164f98a93 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:21.536832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09181896-02d3-44e4-9497-78f164f98a93 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:21.536942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09181896-02d3-44e4-9497-78f164f98a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:21.555274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-09181896-02d3-44e4-9497-78f164f98a93 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:21.556828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-09181896-02d3-44e4-9497-78f164f98a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:21.624053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 23/48] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:21.832545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f1974-5f8a-49e4-bf48-2d2501bf2667 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:21.832829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f1974-5f8a-49e4-bf48-2d2501bf2667 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:21.833101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f1974-5f8a-49e4-bf48-2d2501bf2667 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:21.834434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 26/49] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:21.839757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3858fb-f92a-4114-bbd7-ece988daf294 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:21.840052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3858fb-f92a-4114-bbd7-ece988daf294 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:21.840366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3858fb-f92a-4114-bbd7-ece988daf294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:21.858766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-ce3858fb-f92a-4114-bbd7-ece988daf294 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:21.859949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ce3858fb-f92a-4114-bbd7-ece988daf294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:21.885845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:21.897431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f1974-5f8a-49e4-bf48-2d2501bf2667 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:21.926987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 24/50] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:22.003843 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f1974-5f8a-49e4-bf48-2d2501bf2667 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:22.021215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:22.038089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4b304-b51e-48d7-a780-55a10f1f5114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:22.038728 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4b304-b51e-48d7-a780-55a10f1f5114 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:22.039014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4b304-b51e-48d7-a780-55a10f1f5114 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:22.039294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4b304-b51e-48d7-a780-55a10f1f5114 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:22.050747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4b304-b51e-48d7-a780-55a10f1f5114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-8cc4b304-b51e-48d7-a780-55a10f1f5114 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:22.055980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:22.057012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:22.057803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 27/51] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:39:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:39:22.063031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3858fb-f92a-4114-bbd7-ece988daf294 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:22.162021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3858fb-f92a-4114-bbd7-ece988daf294 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:22.195900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4b304-b51e-48d7-a780-55a10f1f5114 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:22.196141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42462095-f3f0-4c4a-b15e-4c3eb7b97048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:22.196799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42462095-f3f0-4c4a-b15e-4c3eb7b97048 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:22.197081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42462095-f3f0-4c4a-b15e-4c3eb7b97048 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:22.197356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42462095-f3f0-4c4a-b15e-4c3eb7b97048 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:22.198032 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42462095-f3f0-4c4a-b15e-4c3eb7b97048 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:22.198336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42462095-f3f0-4c4a-b15e-4c3eb7b97048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:22.199761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42462095-f3f0-4c4a-b15e-4c3eb7b97048 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:22.209065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 25/52] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:22.214744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42462095-f3f0-4c4a-b15e-4c3eb7b97048 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:22.243390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:22.298466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4b304-b51e-48d7-a780-55a10f1f5114 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:22.337248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79db02-c9f0-42d0-aca7-fc679eebd838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:22.337797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79db02-c9f0-42d0-aca7-fc679eebd838 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:22.338082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79db02-c9f0-42d0-aca7-fc679eebd838 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:22.338359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79db02-c9f0-42d0-aca7-fc679eebd838 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:22.339624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79db02-c9f0-42d0-aca7-fc679eebd838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:22.341194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79db02-c9f0-42d0-aca7-fc679eebd838 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:22.342902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 28/53] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:22.348810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79db02-c9f0-42d0-aca7-fc679eebd838 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:22.400616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:22.419725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42462095-f3f0-4c4a-b15e-4c3eb7b97048 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:22.454708 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79db02-c9f0-42d0-aca7-fc679eebd838 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:22.455470 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d46617-6354-479d-a536-2e6de2a8ea9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:22.456109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d46617-6354-479d-a536-2e6de2a8ea9b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:22.456476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d46617-6354-479d-a536-2e6de2a8ea9b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:22.456755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d46617-6354-479d-a536-2e6de2a8ea9b None admin] PATH_INFO: `/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:22.457915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d46617-6354-479d-a536-2e6de2a8ea9b None admin] RBAC: Authorizing `identity:get_user(user_id=374ef1a08ea74e7cb0bf480e25c8cdd4)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:22.459365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d46617-6354-479d-a536-2e6de2a8ea9b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:22.461000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 26/54] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:22 2022] GET /identity/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:22.466514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d46617-6354-479d-a536-2e6de2a8ea9b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:22.490335 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f78e44-e781-4d7b-b9f0-3888c40e9b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:22.490943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f78e44-e781-4d7b-b9f0-3888c40e9b28 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:22.491226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f78e44-e781-4d7b-b9f0-3888c40e9b28 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:22.491505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f78e44-e781-4d7b-b9f0-3888c40e9b28 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:22.492158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f78e44-e781-4d7b-b9f0-3888c40e9b28 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:22.493639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f78e44-e781-4d7b-b9f0-3888c40e9b28 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:22.557326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:22.564232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d46617-6354-479d-a536-2e6de2a8ea9b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:22.596725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040a677-be2d-4f87-bc02-064041fae778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:22.597350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040a677-be2d-4f87-bc02-064041fae778 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:22.599525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040a677-be2d-4f87-bc02-064041fae778 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:22.599525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040a677-be2d-4f87-bc02-064041fae778 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:22.599674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 29/55] 10.222.0.15 () {60 vars in 1140 bytes} [Tue Oct 4 11:39:22 2022] POST /identity/v3/projects => generated 298 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:39:22.600080 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040a677-be2d-4f87-bc02-064041fae778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:22.600530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040a677-be2d-4f87-bc02-064041fae778 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:22.602145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 27/56] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:22.608581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f78e44-e781-4d7b-b9f0-3888c40e9b28 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:22.714581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:22.718677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f78e44-e781-4d7b-b9f0-3888c40e9b28 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:22.756537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f146c0c-1178-486e-b6ce-51a242ce27ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:22.757163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f146c0c-1178-486e-b6ce-51a242ce27ab None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:22.757445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f146c0c-1178-486e-b6ce-51a242ce27ab None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:22.757722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f146c0c-1178-486e-b6ce-51a242ce27ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:22.758460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f146c0c-1178-486e-b6ce-51a242ce27ab None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.domain.id=default, user.id=374ef1a08ea74e7cb0bf480e25c8cdd4) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:22.758773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f146c0c-1178-486e-b6ce-51a242ce27ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:22.760485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f146c0c-1178-486e-b6ce-51a242ce27ab None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:22.766742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 30/57] 10.222.0.15 () {56 vars in 1322 bytes} [Tue Oct 4 11:39:22 2022] GET /identity/v3/role_assignments?user.id=374ef1a08ea74e7cb0bf480e25c8cdd4&scope.domain.id=default&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:24.830165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369b64-a0c5-4293-b80e-2ff934e698f6 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:24.830165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369b64-a0c5-4293-b80e-2ff934e698f6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:24.830165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369b64-a0c5-4293-b80e-2ff934e698f6 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:24.830165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 28/58] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:39:24.835658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dccd73-34fc-4ecd-aab0-19f56d90bf83 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:24.836163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dccd73-34fc-4ecd-aab0-19f56d90bf83 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:24.836291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dccd73-34fc-4ecd-aab0-19f56d90bf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:24.856418 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d5dccd73-34fc-4ecd-aab0-19f56d90bf83 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:24.858733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d5dccd73-34fc-4ecd-aab0-19f56d90bf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:24.930696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 31/59] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:25.206203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53369b64-a0c5-4293-b80e-2ff934e698f6 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:25.312677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53369b64-a0c5-4293-b80e-2ff934e698f6 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:25.348689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106bc07-4516-497c-bb5e-5bcfc93f1534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:25.349325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106bc07-4516-497c-bb5e-5bcfc93f1534 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:25.349612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106bc07-4516-497c-bb5e-5bcfc93f1534 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:25.349892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106bc07-4516-497c-bb5e-5bcfc93f1534 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:25.351119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106bc07-4516-497c-bb5e-5bcfc93f1534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:25.352714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106bc07-4516-497c-bb5e-5bcfc93f1534 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:25.354327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 29/60] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:25.360874 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dccd73-34fc-4ecd-aab0-19f56d90bf83 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:25.457841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:25.466276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dccd73-34fc-4ecd-aab0-19f56d90bf83 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:25.502090 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a671fc6-6605-4825-b8b2-c9086cd35d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:25.502705 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a671fc6-6605-4825-b8b2-c9086cd35d81 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:25.502983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a671fc6-6605-4825-b8b2-c9086cd35d81 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:25.503254 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a671fc6-6605-4825-b8b2-c9086cd35d81 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:25.503917 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a671fc6-6605-4825-b8b2-c9086cd35d81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:25.505416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a671fc6-6605-4825-b8b2-c9086cd35d81 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:25.609867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 32/61] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:39:25 2022] POST /identity/v3/users => generated 275 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:39:29.037684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf77c0-12cb-426f-8449-86abbbed726c None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:29.038699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf77c0-12cb-426f-8449-86abbbed726c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:29.039159 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf77c0-12cb-426f-8449-86abbbed726c None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:29.040802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 30/62] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:29.050023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b3cb8-30ca-4149-be27-b55c94b5557f None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:29.050925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b3cb8-30ca-4149-be27-b55c94b5557f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:29.051386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b3cb8-30ca-4149-be27-b55c94b5557f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:29.080847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-914b3cb8-30ca-4149-be27-b55c94b5557f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:29.082424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-914b3cb8-30ca-4149-be27-b55c94b5557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:29.177344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 33/63] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:29.202208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0eff4-ff92-443c-bade-7e901344fb22 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:29.202861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0eff4-ff92-443c-bade-7e901344fb22 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:29.203354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0eff4-ff92-443c-bade-7e901344fb22 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:29.204992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 31/64] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:29.210868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daecf2-613b-4696-928f-f762fe9d05f2 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:29.211174 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daecf2-613b-4696-928f-f762fe9d05f2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:29.211443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daecf2-613b-4696-928f-f762fe9d05f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:29.237076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b2daecf2-613b-4696-928f-f762fe9d05f2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:29.238294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b2daecf2-613b-4696-928f-f762fe9d05f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:29.310969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cadbe-af2e-43fd-ae0d-8fc60f6ba78c None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:29.311600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cadbe-af2e-43fd-ae0d-8fc60f6ba78c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:29.312081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cadbe-af2e-43fd-ae0d-8fc60f6ba78c None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:29.313683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 32/65] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:39:29.319576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f73afa-5497-4969-b3a6-40af455d449f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:29.320144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f73afa-5497-4969-b3a6-40af455d449f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:29.320688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f73afa-5497-4969-b3a6-40af455d449f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:29.345811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-60f73afa-5497-4969-b3a6-40af455d449f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:29.347550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-60f73afa-5497-4969-b3a6-40af455d449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:29.360224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 34/66] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:29.388972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:29.437620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 33/67] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:29.489272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2daecf2-613b-4696-928f-f762fe9d05f2 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:29.501756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:29.633888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2daecf2-613b-4696-928f-f762fe9d05f2 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:29.683746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30d898-de54-4e10-ac26-c26f95ae8fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:29.684875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30d898-de54-4e10-ac26-c26f95ae8fae None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:29.685441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30d898-de54-4e10-ac26-c26f95ae8fae None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:29.685913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30d898-de54-4e10-ac26-c26f95ae8fae None admin] PATH_INFO: `/v3/roles/member` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:29.700226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30d898-de54-4e10-ac26-c26f95ae8fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-ba30d898-de54-4e10-ac26-c26f95ae8fae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:29.706141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:29.707064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:29.707851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:29.710538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 35/68] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:29.717373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f73afa-5497-4969-b3a6-40af455d449f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:29.761211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30d898-de54-4e10-ac26-c26f95ae8fae None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:29.890141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f73afa-5497-4969-b3a6-40af455d449f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:29.905533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:29.946883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51028dc9-8674-4503-8941-781c28086627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:29.948151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51028dc9-8674-4503-8941-781c28086627 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:29.948662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51028dc9-8674-4503-8941-781c28086627 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:29.949155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51028dc9-8674-4503-8941-781c28086627 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:29.950208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51028dc9-8674-4503-8941-781c28086627 None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:29.950763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51028dc9-8674-4503-8941-781c28086627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:29.952405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51028dc9-8674-4503-8941-781c28086627 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:29.955604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30d898-de54-4e10-ac26-c26f95ae8fae None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:29.963421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 34/69] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:29.964685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51028dc9-8674-4503-8941-781c28086627 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:29.995606 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:30.010550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12438dff-b6e8-4c8e-8faa-05d9814e4354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:30.011555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12438dff-b6e8-4c8e-8faa-05d9814e4354 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.011555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12438dff-b6e8-4c8e-8faa-05d9814e4354 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.011732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12438dff-b6e8-4c8e-8faa-05d9814e4354 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.024339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12438dff-b6e8-4c8e-8faa-05d9814e4354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-12438dff-b6e8-4c8e-8faa-05d9814e4354 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:30.030043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:30.031018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 36/70] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:39:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:30.033033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12438dff-b6e8-4c8e-8faa-05d9814e4354 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:30.095890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51028dc9-8674-4503-8941-781c28086627 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:30.133514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b4188-065b-44a6-8897-3f7525c6f068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:30.134176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b4188-065b-44a6-8897-3f7525c6f068 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.134460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b4188-065b-44a6-8897-3f7525c6f068 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.134743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b4188-065b-44a6-8897-3f7525c6f068 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.145972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b4188-065b-44a6-8897-3f7525c6f068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-f35b4188-065b-44a6-8897-3f7525c6f068 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:30.150928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:30.152197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:30.153612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 35/71] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:39:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:30.154862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89930e-1781-4237-b696-492ff4a2435f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.154862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89930e-1781-4237-b696-492ff4a2435f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.155159 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89930e-1781-4237-b696-492ff4a2435f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.160369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 36/72] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:30.161278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89930e-1781-4237-b696-492ff4a2435f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:30.166830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12438dff-b6e8-4c8e-8faa-05d9814e4354 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:30.197572 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:30.206756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2ddb-7aa7-4e72-9472-22b24785da7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:30.212295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2ddb-7aa7-4e72-9472-22b24785da7c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.212295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2ddb-7aa7-4e72-9472-22b24785da7c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.212295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2ddb-7aa7-4e72-9472-22b24785da7c None admin] PATH_INFO: `/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.224856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2ddb-7aa7-4e72-9472-22b24785da7c None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf9cae9ba447d4851f472b6e46e3fd)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:30.226421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2ddb-7aa7-4e72-9472-22b24785da7c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:30.228147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 37/73] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:30 2022] GET /identity/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:30.229362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb08d1a-9244-4ec5-9a71-f4bd4cc1b11f None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.229644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb08d1a-9244-4ec5-9a71-f4bd4cc1b11f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.229991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb08d1a-9244-4ec5-9a71-f4bd4cc1b11f None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.231703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 38/74] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:30.233462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb08d1a-9244-4ec5-9a71-f4bd4cc1b11f None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:30.266389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:30.286503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89930e-1781-4237-b696-492ff4a2435f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:30.335200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951b2c3-a592-4801-84b0-75b7d912de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:30.335818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951b2c3-a592-4801-84b0-75b7d912de0e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.336103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951b2c3-a592-4801-84b0-75b7d912de0e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.336407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951b2c3-a592-4801-84b0-75b7d912de0e None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.337081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951b2c3-a592-4801-84b0-75b7d912de0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:30.337390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951b2c3-a592-4801-84b0-75b7d912de0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:30.338836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951b2c3-a592-4801-84b0-75b7d912de0e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:30.347521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 37/75] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:30.348940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b9e39-2f2c-4fd5-bfbc-dadfe4d51bb8 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.349236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b9e39-2f2c-4fd5-bfbc-dadfe4d51bb8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.349546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b9e39-2f2c-4fd5-bfbc-dadfe4d51bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.356098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb08d1a-9244-4ec5-9a71-f4bd4cc1b11f None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:30.369292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-370b9e39-2f2c-4fd5-bfbc-dadfe4d51bb8 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:30.370511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-370b9e39-2f2c-4fd5-bfbc-dadfe4d51bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:30.393822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1b387-8892-4a37-8079-0712a8e72936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:30.394453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1b387-8892-4a37-8079-0712a8e72936 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.394724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1b387-8892-4a37-8079-0712a8e72936 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.394992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1b387-8892-4a37-8079-0712a8e72936 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.395668 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1b387-8892-4a37-8079-0712a8e72936 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:30.396038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1b387-8892-4a37-8079-0712a8e72936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:30.397424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1b387-8892-4a37-8079-0712a8e72936 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:30.409334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 39/76] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:39:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:30.410163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1b387-8892-4a37-8079-0712a8e72936 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:30.456198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 38/77] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:30.456198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e444c0-6acd-4be0-8320-b1e1bcfcc80d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.456198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e444c0-6acd-4be0-8320-b1e1bcfcc80d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.456198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e444c0-6acd-4be0-8320-b1e1bcfcc80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.463312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-a8e444c0-6acd-4be0-8320-b1e1bcfcc80d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:30.464550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a8e444c0-6acd-4be0-8320-b1e1bcfcc80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:30.480506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:30.536431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1b387-8892-4a37-8079-0712a8e72936 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:30.538439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 39/78] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:30.539302 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e444c0-6acd-4be0-8320-b1e1bcfcc80d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:30.548717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:30.585130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53082587-ce22-4c4e-b98b-4fdcb069e332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:30.585795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082587-ce22-4c4e-b98b-4fdcb069e332 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.586079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082587-ce22-4c4e-b98b-4fdcb069e332 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.586373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082587-ce22-4c4e-b98b-4fdcb069e332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.648899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e444c0-6acd-4be0-8320-b1e1bcfcc80d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:30.680162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53082587-ce22-4c4e-b98b-4fdcb069e332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:30.683048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53082587-ce22-4c4e-b98b-4fdcb069e332 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:30.685446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdeedb4-ec4d-4c5e-928d-592b1318e5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:30.686097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeedb4-ec4d-4c5e-928d-592b1318e5fa None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.686414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeedb4-ec4d-4c5e-928d-592b1318e5fa None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.686791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeedb4-ec4d-4c5e-928d-592b1318e5fa None admin] PATH_INFO: `/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.687987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdeedb4-ec4d-4c5e-928d-592b1318e5fa None admin] RBAC: Authorizing `identity:get_user(user_id=374ef1a08ea74e7cb0bf480e25c8cdd4)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:30.689555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdeedb4-ec4d-4c5e-928d-592b1318e5fa None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:30.691962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 40/79] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:30 2022] GET /identity/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:30.693272 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdeedb4-ec4d-4c5e-928d-592b1318e5fa None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:30.752371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:30.808212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:30.839839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdeedb4-ec4d-4c5e-928d-592b1318e5fa None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:30.843337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 40/80] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:30.844726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53082587-ce22-4c4e-b98b-4fdcb069e332 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:30.876111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d5aba-28cd-4cac-84b9-924dddc6baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:30.876884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d5aba-28cd-4cac-84b9-924dddc6baae None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.877232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d5aba-28cd-4cac-84b9-924dddc6baae None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.877630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d5aba-28cd-4cac-84b9-924dddc6baae None admin] PATH_INFO: `/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:30.878820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d5aba-28cd-4cac-84b9-924dddc6baae None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf9cae9ba447d4851f472b6e46e3fd)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:30.880334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d5aba-28cd-4cac-84b9-924dddc6baae None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:30.881980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 41/81] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:30 2022] GET /identity/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:30.882747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d5aba-28cd-4cac-84b9-924dddc6baae None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:30.964580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53082587-ce22-4c4e-b98b-4fdcb069e332 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:30.987373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d5aba-28cd-4cac-84b9-924dddc6baae None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:30.998543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c79ad-287f-4609-b74c-6a25df6c9079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:30.999156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c79ad-287f-4609-b74c-6a25df6c9079 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:30.999436 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c79ad-287f-4609-b74c-6a25df6c9079 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:30.999713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c79ad-287f-4609-b74c-6a25df6c9079 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:31.018351 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:31.023834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a115e9d-52f1-41f9-a245-0a490cf7bc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:31.024509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a115e9d-52f1-41f9-a245-0a490cf7bc05 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:31.028402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a115e9d-52f1-41f9-a245-0a490cf7bc05 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:31.028402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a115e9d-52f1-41f9-a245-0a490cf7bc05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:31.028402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a115e9d-52f1-41f9-a245-0a490cf7bc05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:31.028402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a115e9d-52f1-41f9-a245-0a490cf7bc05 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:31.030106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 42/82] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:31.030932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a115e9d-52f1-41f9-a245-0a490cf7bc05 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:31.042684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:31.076559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c79ad-287f-4609-b74c-6a25df6c9079 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:31.078069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c79ad-287f-4609-b74c-6a25df6c9079 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:31.137187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a115e9d-52f1-41f9-a245-0a490cf7bc05 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:31.176967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793a492-234b-477e-9fc2-4bfe1802f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:31.177614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a492-234b-477e-9fc2-4bfe1802f316 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:31.177900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a492-234b-477e-9fc2-4bfe1802f316 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:31.178186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a492-234b-477e-9fc2-4bfe1802f316 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:31.187340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 41/83] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:31.188220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c79ad-287f-4609-b74c-6a25df6c9079 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:31.190505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793a492-234b-477e-9fc2-4bfe1802f316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-9793a492-234b-477e-9fc2-4bfe1802f316 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:31.195633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:31.197170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 43/84] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:31.198429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793a492-234b-477e-9fc2-4bfe1802f316 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:31.245330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:31.271004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:31.301224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c79ad-287f-4609-b74c-6a25df6c9079 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:31.315957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793a492-234b-477e-9fc2-4bfe1802f316 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:31.337372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b9d00-6e59-405b-8c73-8fd67e2218a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:31.338010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b9d00-6e59-405b-8c73-8fd67e2218a1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:31.338302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b9d00-6e59-405b-8c73-8fd67e2218a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:31.338579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b9d00-6e59-405b-8c73-8fd67e2218a1 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:31.339727 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b9d00-6e59-405b-8c73-8fd67e2218a1 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:31.341230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b9d00-6e59-405b-8c73-8fd67e2218a1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:31.342911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 42/85] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:31 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:31.343745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b9d00-6e59-405b-8c73-8fd67e2218a1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:31.363076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cc1bb-81ec-4eb2-bd0c-631ceacecb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:31.363809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cc1bb-81ec-4eb2-bd0c-631ceacecb69 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:31.364101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cc1bb-81ec-4eb2-bd0c-631ceacecb69 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:31.364416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cc1bb-81ec-4eb2-bd0c-631ceacecb69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:31.439277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cc1bb-81ec-4eb2-bd0c-631ceacecb69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:31.440805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cc1bb-81ec-4eb2-bd0c-631ceacecb69 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:31.469239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b9d00-6e59-405b-8c73-8fd67e2218a1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:31.491820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:31.510867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:31.522474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e60e1-a56f-42d3-9c1e-da3efc0e8558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:31.523188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e60e1-a56f-42d3-9c1e-da3efc0e8558 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:31.523352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e60e1-a56f-42d3-9c1e-da3efc0e8558 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:31.523624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e60e1-a56f-42d3-9c1e-da3efc0e8558 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:31.524310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e60e1-a56f-42d3-9c1e-da3efc0e8558 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:31.525818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e60e1-a56f-42d3-9c1e-da3efc0e8558 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:31.553819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 44/86] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:31.554692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cc1bb-81ec-4eb2-bd0c-631ceacecb69 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:31.619811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 43/87] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:39:31 2022] POST /identity/v3/users => generated 289 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:39:31.621159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e60e1-a56f-42d3-9c1e-da3efc0e8558 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:31.662395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cc1bb-81ec-4eb2-bd0c-631ceacecb69 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:31.707637 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3f5e9-e734-447a-88bc-d25a6661e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:31.708326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3f5e9-e734-447a-88bc-d25a6661e639 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:31.708610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3f5e9-e734-447a-88bc-d25a6661e639 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:31.708885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3f5e9-e734-447a-88bc-d25a6661e639 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:31.710101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3f5e9-e734-447a-88bc-d25a6661e639 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:31.711584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3f5e9-e734-447a-88bc-d25a6661e639 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:31.713336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 45/88] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:31 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:31.714239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3f5e9-e734-447a-88bc-d25a6661e639 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:31.771953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:31.791735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e60e1-a56f-42d3-9c1e-da3efc0e8558 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:31.857504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe8310-bc16-4e61-b75e-c2f7e4bbe617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:31.858196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe8310-bc16-4e61-b75e-c2f7e4bbe617 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:31.858412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe8310-bc16-4e61-b75e-c2f7e4bbe617 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:31.858717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe8310-bc16-4e61-b75e-c2f7e4bbe617 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:31.859360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe8310-bc16-4e61-b75e-c2f7e4bbe617 None admin] RBAC: Adding query filter params (name=member) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:31.861190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe8310-bc16-4e61-b75e-c2f7e4bbe617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:31.861190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe8310-bc16-4e61-b75e-c2f7e4bbe617 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:31.872069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 44/89] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:31.873220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe8310-bc16-4e61-b75e-c2f7e4bbe617 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:31.878181 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3f5e9-e734-447a-88bc-d25a6661e639 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:31.935320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd8e0a-8d84-4968-b784-20381a4d7790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:31.936272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:31.936530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8e0a-8d84-4968-b784-20381a4d7790 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:31.936778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8e0a-8d84-4968-b784-20381a4d7790 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:31.937061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd8e0a-8d84-4968-b784-20381a4d7790 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:31.937814 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd8e0a-8d84-4968-b784-20381a4d7790 None admin] RBAC: Adding query filter params (role.id=3f0c34a1f93e49adb1757f315333cc48, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c, user.id=c0bf9cae9ba447d4851f472b6e46e3fd) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:31.938122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd8e0a-8d84-4968-b784-20381a4d7790 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:31.940835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd8e0a-8d84-4968-b784-20381a4d7790 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:31.956964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 46/90] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:31 2022] GET /identity/v3/role_assignments?user.id=c0bf9cae9ba447d4851f472b6e46e3fd&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:31.956964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd8e0a-8d84-4968-b784-20381a4d7790 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:32.077842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe8310-bc16-4e61-b75e-c2f7e4bbe617 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:32.151235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb28fc-2fda-4e75-a65f-91b2890071e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:32.152334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb28fc-2fda-4e75-a65f-91b2890071e2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:32.152628 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb28fc-2fda-4e75-a65f-91b2890071e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:32.153174 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb28fc-2fda-4e75-a65f-91b2890071e2 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:32.153278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:32.155009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb28fc-2fda-4e75-a65f-91b2890071e2 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:32.158419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb28fc-2fda-4e75-a65f-91b2890071e2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:32.158753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 45/91] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:31 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:32.159649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb28fc-2fda-4e75-a65f-91b2890071e2 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:32.165695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd8e0a-8d84-4968-b784-20381a4d7790 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:32.213934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b516f-2405-428c-a481-0c5d421f3152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:32.214566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b516f-2405-428c-a481-0c5d421f3152 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:32.214853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b516f-2405-428c-a481-0c5d421f3152 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:32.215134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b516f-2405-428c-a481-0c5d421f3152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:32.215879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b516f-2405-428c-a481-0c5d421f3152 None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c, user.id=374ef1a08ea74e7cb0bf480e25c8cdd4) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:32.216857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b516f-2405-428c-a481-0c5d421f3152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:32.218422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b516f-2405-428c-a481-0c5d421f3152 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:32.226413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 47/92] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:31 2022] GET /identity/v3/role_assignments?user.id=374ef1a08ea74e7cb0bf480e25c8cdd4&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:32.227588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b516f-2405-428c-a481-0c5d421f3152 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:32.308718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb28fc-2fda-4e75-a65f-91b2890071e2 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:32.313647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:32.363691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92cbd1-7ac4-4bc9-8822-3f0d61a85286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:32.366014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92cbd1-7ac4-4bc9-8822-3f0d61a85286 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:32.369103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92cbd1-7ac4-4bc9-8822-3f0d61a85286 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:32.369541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92cbd1-7ac4-4bc9-8822-3f0d61a85286 None admin] PATH_INFO: `/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:32.373511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92cbd1-7ac4-4bc9-8822-3f0d61a85286 None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf9cae9ba447d4851f472b6e46e3fd)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:32.377314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92cbd1-7ac4-4bc9-8822-3f0d61a85286 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:32.380943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 46/93] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:32 2022] GET /identity/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:32.386582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92cbd1-7ac4-4bc9-8822-3f0d61a85286 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:32.452557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b516f-2405-428c-a481-0c5d421f3152 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:32.560963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622abcb8-5f28-4148-99e8-61323f1115e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:32.561589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622abcb8-5f28-4148-99e8-61323f1115e4 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:32.561907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622abcb8-5f28-4148-99e8-61323f1115e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:32.562192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622abcb8-5f28-4148-99e8-61323f1115e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:32.562940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622abcb8-5f28-4148-99e8-61323f1115e4 None admin] RBAC: Adding query filter params (role.id=4bb04c67143448539c5af8d8bdd621f9, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c, user.id=c0bf9cae9ba447d4851f472b6e46e3fd) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:32.563255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622abcb8-5f28-4148-99e8-61323f1115e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:32.564824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622abcb8-5f28-4148-99e8-61323f1115e4 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:32.577896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 48/94] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:32 2022] GET /identity/v3/role_assignments?user.id=c0bf9cae9ba447d4851f472b6e46e3fd&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=4bb04c67143448539c5af8d8bdd621f9 => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:32.608591 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92cbd1-7ac4-4bc9-8822-3f0d61a85286 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:32.657596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffe04a-830f-49a4-b698-d978c01be92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:32.658530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffe04a-830f-49a4-b698-d978c01be92d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:32.659010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffe04a-830f-49a4-b698-d978c01be92d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:32.659541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffe04a-830f-49a4-b698-d978c01be92d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:32.714922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:32.760310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffe04a-830f-49a4-b698-d978c01be92d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:32.762192 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffe04a-830f-49a4-b698-d978c01be92d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:32.908590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 47/95] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:32.914891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622abcb8-5f28-4148-99e8-61323f1115e4 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:32.965251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:33.062316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622abcb8-5f28-4148-99e8-61323f1115e4 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:33.115479 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126f36b-db42-4aa1-ae82-069b2e9badea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:33.116341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126f36b-db42-4aa1-ae82-069b2e9badea None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:33.116802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126f36b-db42-4aa1-ae82-069b2e9badea None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:33.117241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126f36b-db42-4aa1-ae82-069b2e9badea None admin] PATH_INFO: `/v3/projects/883884da6bf64da0afd09d6d4956d7f2` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:33.118928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126f36b-db42-4aa1-ae82-069b2e9badea None admin] RBAC: Authorizing `identity:get_project(project_id=883884da6bf64da0afd09d6d4956d7f2)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:33.121376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126f36b-db42-4aa1-ae82-069b2e9badea None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:33.124730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 49/96] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:32 2022] GET /identity/v3/projects/883884da6bf64da0afd09d6d4956d7f2 => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:33.132221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffe04a-830f-49a4-b698-d978c01be92d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:33.245674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:33.302433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffe04a-830f-49a4-b698-d978c01be92d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:33.362413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02813f87-e6ee-4d9f-93c4-91b4dee7e78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:33.363112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02813f87-e6ee-4d9f-93c4-91b4dee7e78c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:33.363384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02813f87-e6ee-4d9f-93c4-91b4dee7e78c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:33.363666 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02813f87-e6ee-4d9f-93c4-91b4dee7e78c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:33.364444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02813f87-e6ee-4d9f-93c4-91b4dee7e78c None admin] RBAC: Adding query filter params (role.id=3f0c34a1f93e49adb1757f315333cc48, scope.project.id=883884da6bf64da0afd09d6d4956d7f2, user.id=c0bf9cae9ba447d4851f472b6e46e3fd) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:33.364763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02813f87-e6ee-4d9f-93c4-91b4dee7e78c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:33.366236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02813f87-e6ee-4d9f-93c4-91b4dee7e78c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:33.376845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 48/97] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:33 2022] GET /identity/v3/role_assignments?user.id=c0bf9cae9ba447d4851f472b6e46e3fd&scope.project.id=883884da6bf64da0afd09d6d4956d7f2&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:35.020668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379aacff-424e-4a9a-986c-ca28eaca9e21 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:35.021719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379aacff-424e-4a9a-986c-ca28eaca9e21 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:35.022210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379aacff-424e-4a9a-986c-ca28eaca9e21 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:35.023781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 50/98] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:35.029535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068b7b8-1f87-45b5-aa8f-a1541f90eece None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:35.030011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068b7b8-1f87-45b5-aa8f-a1541f90eece None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:35.030263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068b7b8-1f87-45b5-aa8f-a1541f90eece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:35.051255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-1068b7b8-1f87-45b5-aa8f-a1541f90eece None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:35.052426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1068b7b8-1f87-45b5-aa8f-a1541f90eece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:35.138248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 49/99] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:35.441296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379aacff-424e-4a9a-986c-ca28eaca9e21 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:35.613510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:35.627778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379aacff-424e-4a9a-986c-ca28eaca9e21 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:35.669741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1856998-d8e4-4bbb-a98b-4ab5d66ff05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:35.672952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1856998-d8e4-4bbb-a98b-4ab5d66ff05f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:35.673286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1856998-d8e4-4bbb-a98b-4ab5d66ff05f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:35.673516 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1856998-d8e4-4bbb-a98b-4ab5d66ff05f None admin] PATH_INFO: `/v3/roles/member` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:35.697215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1856998-d8e4-4bbb-a98b-4ab5d66ff05f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-b1856998-d8e4-4bbb-a98b-4ab5d66ff05f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:35.702818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:35.703734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:35.704612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 51/100] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:35.710107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068b7b8-1f87-45b5-aa8f-a1541f90eece None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:35.826654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:35.858160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830fb55-b1cf-49d9-8136-9dd135972920 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:35.858529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830fb55-b1cf-49d9-8136-9dd135972920 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:35.858884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830fb55-b1cf-49d9-8136-9dd135972920 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:35.860527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 52/101] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:35.866711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f20a1-7bf2-478f-b321-e5d782547abb None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:35.867254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f20a1-7bf2-478f-b321-e5d782547abb None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:35.867713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f20a1-7bf2-478f-b321-e5d782547abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:35.883525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068b7b8-1f87-45b5-aa8f-a1541f90eece None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:35.913984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-674f20a1-7bf2-478f-b321-e5d782547abb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:35.915559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-674f20a1-7bf2-478f-b321-e5d782547abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:35.932789 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bd155-7ae9-4954-b436-8c4249497c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:35.934232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bd155-7ae9-4954-b436-8c4249497c78 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:35.935292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bd155-7ae9-4954-b436-8c4249497c78 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:35.936062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bd155-7ae9-4954-b436-8c4249497c78 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:35.937701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bd155-7ae9-4954-b436-8c4249497c78 None admin] RBAC: Adding query filter params (name=member) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:35.938229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bd155-7ae9-4954-b436-8c4249497c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:35.941398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bd155-7ae9-4954-b436-8c4249497c78 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:35.961497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 50/102] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:35.964524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7154-935c-4070-aea5-38013fd77556 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:35.965026 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7154-935c-4070-aea5-38013fd77556 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:35.965530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7154-935c-4070-aea5-38013fd77556 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:35.969871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 51/103] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:35.976194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc5eb-8a98-4560-bfda-850ac2a8dd7b None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:35.976194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc5eb-8a98-4560-bfda-850ac2a8dd7b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:35.976194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcc5eb-8a98-4560-bfda-850ac2a8dd7b None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:35.976194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 52/104] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:35.976194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcc5eb-8a98-4560-bfda-850ac2a8dd7b None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:36.006064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 53/105] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:39:36.007691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824d38a-b733-4d18-9f7d-62ce110a7485 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.008115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824d38a-b733-4d18-9f7d-62ce110a7485 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.008724 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824d38a-b733-4d18-9f7d-62ce110a7485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.039012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c824d38a-b733-4d18-9f7d-62ce110a7485 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:36.043174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c824d38a-b733-4d18-9f7d-62ce110a7485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:36.187399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 54/106] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:36.189058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18e788-2595-4041-92d6-61d83d4cee9a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.189625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18e788-2595-4041-92d6-61d83d4cee9a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.190082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18e788-2595-4041-92d6-61d83d4cee9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.207608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcc5eb-8a98-4560-bfda-850ac2a8dd7b None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:36.210040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-ee18e788-2595-4041-92d6-61d83d4cee9a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:36.211556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ee18e788-2595-4041-92d6-61d83d4cee9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:36.226585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:36.241596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:36.252310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eec514-d2dd-4b29-89d8-bb150976dfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:36.253363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eec514-d2dd-4b29-89d8-bb150976dfd6 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.253851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eec514-d2dd-4b29-89d8-bb150976dfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.254399 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eec514-d2dd-4b29-89d8-bb150976dfd6 None admin] PATH_INFO: `/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.256028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eec514-d2dd-4b29-89d8-bb150976dfd6 None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf9cae9ba447d4851f472b6e46e3fd)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:36.257806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eec514-d2dd-4b29-89d8-bb150976dfd6 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:36.264271 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 53/107] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:35 2022] GET /identity/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:36.266181 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8f235-890f-4181-adbc-035ad3ca3b9c None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.266686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8f235-890f-4181-adbc-035ad3ca3b9c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.267189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8f235-890f-4181-adbc-035ad3ca3b9c None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.268842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 54/108] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:36.275282 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8f235-890f-4181-adbc-035ad3ca3b9c None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:36.306761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 55/109] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:36.308441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1191cf6-6457-4f44-91f2-e7e8e68cfd79 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.308958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1191cf6-6457-4f44-91f2-e7e8e68cfd79 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.309439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1191cf6-6457-4f44-91f2-e7e8e68cfd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.329241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b1191cf6-6457-4f44-91f2-e7e8e68cfd79 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:36.330860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b1191cf6-6457-4f44-91f2-e7e8e68cfd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:36.403344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 56/110] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:36.409278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4087a-c9e0-4bfd-8ea5-81af8fc34ea3 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.409848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4087a-c9e0-4bfd-8ea5-81af8fc34ea3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.410442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4087a-c9e0-4bfd-8ea5-81af8fc34ea3 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.411960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 57/111] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:36.414118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8f235-890f-4181-adbc-035ad3ca3b9c None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:36.417551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e131128-78c5-4ae4-b29b-b472d13e74cf None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.418080 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e131128-78c5-4ae4-b29b-b472d13e74cf None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.418614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e131128-78c5-4ae4-b29b-b472d13e74cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.451644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-5e131128-78c5-4ae4-b29b-b472d13e74cf None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:36.453227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5e131128-78c5-4ae4-b29b-b472d13e74cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:36.453888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ffbdb-6748-4157-b44a-cfb5d3d20a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:36.454528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ffbdb-6748-4157-b44a-cfb5d3d20a0a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.454810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ffbdb-6748-4157-b44a-cfb5d3d20a0a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.455085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ffbdb-6748-4157-b44a-cfb5d3d20a0a None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.456217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ffbdb-6748-4157-b44a-cfb5d3d20a0a None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:36.458699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ffbdb-6748-4157-b44a-cfb5d3d20a0a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:36.460906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 55/112] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:36 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:36.472805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ffbdb-6748-4157-b44a-cfb5d3d20a0a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:36.556471 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:36.604531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ffbdb-6748-4157-b44a-cfb5d3d20a0a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:36.618528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 58/113] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:36.619868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e131128-78c5-4ae4-b29b-b472d13e74cf None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:36.633300 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:36.666853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c455-1bc6-4dfc-891e-4d63c653f1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:36.667531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c455-1bc6-4dfc-891e-4d63c653f1e3 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.667812 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c455-1bc6-4dfc-891e-4d63c653f1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.668089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c455-1bc6-4dfc-891e-4d63c653f1e3 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/users/c0bf9cae9ba447d4851f472b6e46e3fd/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.674889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2c455-1bc6-4dfc-891e-4d63c653f1e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=973ebf7f6f414136a6ae5a7d1a1a919c, user_id=c0bf9cae9ba447d4851f472b6e46e3fd, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:36.676520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2c455-1bc6-4dfc-891e-4d63c653f1e3 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:36.755627 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e131128-78c5-4ae4-b29b-b472d13e74cf None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:36.759486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 56/114] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:39:36 2022] PUT /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/users/c0bf9cae9ba447d4851f472b6e46e3fd/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 11:39:36.760560 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c455-1bc6-4dfc-891e-4d63c653f1e3 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:36.809351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db181a8-edf8-4e47-b264-91c5427ce070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:36.810856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db181a8-edf8-4e47-b264-91c5427ce070 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.811215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db181a8-edf8-4e47-b264-91c5427ce070 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.811967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db181a8-edf8-4e47-b264-91c5427ce070 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.824971 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db181a8-edf8-4e47-b264-91c5427ce070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-4db181a8-edf8-4e47-b264-91c5427ce070 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:36.830749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:36.831875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:39:36.833258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:36.834688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 59/115] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:36.834833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db181a8-edf8-4e47-b264-91c5427ce070 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:36.916131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c455-1bc6-4dfc-891e-4d63c653f1e3 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:36.924466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:36.943650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:36.959832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4efe8-fb97-472d-924d-181827e6666e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:36.960822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4efe8-fb97-472d-924d-181827e6666e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:36.961246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4efe8-fb97-472d-924d-181827e6666e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:36.966259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4efe8-fb97-472d-924d-181827e6666e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:36.966259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4efe8-fb97-472d-924d-181827e6666e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:36.966259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4efe8-fb97-472d-924d-181827e6666e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:36.966259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 57/116] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:36.967189 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db181a8-edf8-4e47-b264-91c5427ce070 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:36.967518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4efe8-fb97-472d-924d-181827e6666e None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.006634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e535e-e830-4857-9bb8-74bb8535e012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:37.007198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e535e-e830-4857-9bb8-74bb8535e012 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:37.007538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e535e-e830-4857-9bb8-74bb8535e012 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:37.007764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e535e-e830-4857-9bb8-74bb8535e012 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:37.019135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e535e-e830-4857-9bb8-74bb8535e012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-791e535e-e830-4857-9bb8-74bb8535e012 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:37.024284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:37.025311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:37.026340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 60/117] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:39:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:37.027196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e535e-e830-4857-9bb8-74bb8535e012 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.078753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4efe8-fb97-472d-924d-181827e6666e None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:37.115261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3923c-d99c-4f13-a1a0-072b8d07a366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:37.116363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3923c-d99c-4f13-a1a0-072b8d07a366 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:37.116656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3923c-d99c-4f13-a1a0-072b8d07a366 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:37.116935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3923c-d99c-4f13-a1a0-072b8d07a366 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:37.128869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3923c-d99c-4f13-a1a0-072b8d07a366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-68d3923c-d99c-4f13-a1a0-072b8d07a366 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:37.134343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:37.135233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:37.136092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 58/118] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:39:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:37.137640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3923c-d99c-4f13-a1a0-072b8d07a366 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.146572 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e535e-e830-4857-9bb8-74bb8535e012 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:37.175383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:37.186503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d971a5-6572-4c15-b63f-92c3b1e59ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:37.187125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d971a5-6572-4c15-b63f-92c3b1e59ac8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:37.187454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d971a5-6572-4c15-b63f-92c3b1e59ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:37.187756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d971a5-6572-4c15-b63f-92c3b1e59ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:37.188498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d971a5-6572-4c15-b63f-92c3b1e59ac8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:37.188780 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d971a5-6572-4c15-b63f-92c3b1e59ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:37.190464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d971a5-6572-4c15-b63f-92c3b1e59ac8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:37.192376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:37.199967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 61/119] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:37.201382 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d971a5-6572-4c15-b63f-92c3b1e59ac8 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.253451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3923c-d99c-4f13-a1a0-072b8d07a366 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:37.294008 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183d84a-5e19-40e5-aafa-e97222ae5fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:37.295120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183d84a-5e19-40e5-aafa-e97222ae5fe9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:37.295408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183d84a-5e19-40e5-aafa-e97222ae5fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:37.295687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183d84a-5e19-40e5-aafa-e97222ae5fe9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:37.306488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183d84a-5e19-40e5-aafa-e97222ae5fe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-c183d84a-5e19-40e5-aafa-e97222ae5fe9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:37.312107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:37.313092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:37.313890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 59/120] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:37.315488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183d84a-5e19-40e5-aafa-e97222ae5fe9 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.324977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d971a5-6572-4c15-b63f-92c3b1e59ac8 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:37.367748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b011-5a86-4179-880c-2d8933ca4851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:37.368420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b011-5a86-4179-880c-2d8933ca4851 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:37.368712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b011-5a86-4179-880c-2d8933ca4851 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:37.369049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788b011-5a86-4179-880c-2d8933ca4851 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:37.369664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b011-5a86-4179-880c-2d8933ca4851 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:37.371155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788b011-5a86-4179-880c-2d8933ca4851 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:37.424424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183d84a-5e19-40e5-aafa-e97222ae5fe9 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:37.435184 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:37.459318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f6862-ead6-470d-95b3-5f99846ca64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:37.459931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f6862-ead6-470d-95b3-5f99846ca64c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:37.460240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f6862-ead6-470d-95b3-5f99846ca64c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:37.460525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f6862-ead6-470d-95b3-5f99846ca64c None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:37.461198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f6862-ead6-470d-95b3-5f99846ca64c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:37.461509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f6862-ead6-470d-95b3-5f99846ca64c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:37.462935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f6862-ead6-470d-95b3-5f99846ca64c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:37.470233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 62/121] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:39:37 2022] POST /identity/v3/projects => generated 302 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:39:37.471842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b011-5a86-4179-880c-2d8933ca4851 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.473838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 60/122] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:37.474741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f6862-ead6-470d-95b3-5f99846ca64c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.532998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:37.597206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f6862-ead6-470d-95b3-5f99846ca64c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:37.608293 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788b011-5a86-4179-880c-2d8933ca4851 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:37.635681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bc7d6-92ad-4368-b995-a9cd61dbd9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:37.636364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bc7d6-92ad-4368-b995-a9cd61dbd9ac None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:37.636644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bc7d6-92ad-4368-b995-a9cd61dbd9ac None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:37.636918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bc7d6-92ad-4368-b995-a9cd61dbd9ac None admin] PATH_INFO: `/v3/users/d972b329d9a74fdaac32d9cf48c2e9dc` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:37.654120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bc7d6-92ad-4368-b995-a9cd61dbd9ac None admin] RBAC: Authorizing `identity:get_user(user_id=d972b329d9a74fdaac32d9cf48c2e9dc)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:37.655746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bc7d6-92ad-4368-b995-a9cd61dbd9ac None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:37.657984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 61/123] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:37 2022] GET /identity/v3/users/d972b329d9a74fdaac32d9cf48c2e9dc => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:37.658675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bc7d6-92ad-4368-b995-a9cd61dbd9ac None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.665074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71a9cb-5914-4897-a344-149fea99cc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:37.666324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a9cb-5914-4897-a344-149fea99cc93 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:37.666617 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a9cb-5914-4897-a344-149fea99cc93 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:37.667018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a9cb-5914-4897-a344-149fea99cc93 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:37.667859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71a9cb-5914-4897-a344-149fea99cc93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:37.668209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71a9cb-5914-4897-a344-149fea99cc93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:37.670268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71a9cb-5914-4897-a344-149fea99cc93 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:37.679589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 63/124] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:39:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:37.684575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71a9cb-5914-4897-a344-149fea99cc93 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.714413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:37.806395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bc7d6-92ad-4368-b995-a9cd61dbd9ac None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:37.827502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71a9cb-5914-4897-a344-149fea99cc93 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:37.829389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:37.846342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fb84f-4fa2-49f3-87bf-f333d74fc38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:37.847010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fb84f-4fa2-49f3-87bf-f333d74fc38a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:37.847355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fb84f-4fa2-49f3-87bf-f333d74fc38a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:37.847577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fb84f-4fa2-49f3-87bf-f333d74fc38a None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:37.848279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fb84f-4fa2-49f3-87bf-f333d74fc38a None admin] RBAC: Adding query filter params (name=member) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:37.848591 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fb84f-4fa2-49f3-87bf-f333d74fc38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:37.849985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fb84f-4fa2-49f3-87bf-f333d74fc38a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:37.861700 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 62/125] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:37.862452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fb84f-4fa2-49f3-87bf-f333d74fc38a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.876410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce639e-7669-4ca9-8bbd-d81896b40d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:37.878936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce639e-7669-4ca9-8bbd-d81896b40d60 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:37.879213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce639e-7669-4ca9-8bbd-d81896b40d60 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:37.879491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce639e-7669-4ca9-8bbd-d81896b40d60 None admin] PATH_INFO: `/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:37.881861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce639e-7669-4ca9-8bbd-d81896b40d60 None admin] RBAC: Authorizing `identity:get_user(user_id=374ef1a08ea74e7cb0bf480e25c8cdd4)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:37.883335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce639e-7669-4ca9-8bbd-d81896b40d60 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:37.885138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 64/126] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:37 2022] GET /identity/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:37.886625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce639e-7669-4ca9-8bbd-d81896b40d60 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:37.992840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:38.003586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fb84f-4fa2-49f3-87bf-f333d74fc38a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:38.023008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce639e-7669-4ca9-8bbd-d81896b40d60 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:38.047202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d65eed-41b2-49a7-ba74-f4a9a17e3d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:38.048108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d65eed-41b2-49a7-ba74-f4a9a17e3d26 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:38.048427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d65eed-41b2-49a7-ba74-f4a9a17e3d26 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:38.048706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d65eed-41b2-49a7-ba74-f4a9a17e3d26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:38.070881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3718c-ecac-4d0d-a3b2-1846e47d6446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:38.072130 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3718c-ecac-4d0d-a3b2-1846e47d6446 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:38.073127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3718c-ecac-4d0d-a3b2-1846e47d6446 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:38.073614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3718c-ecac-4d0d-a3b2-1846e47d6446 None admin] PATH_INFO: `/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:38.075005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3718c-ecac-4d0d-a3b2-1846e47d6446 None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf9cae9ba447d4851f472b6e46e3fd)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:38.078162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3718c-ecac-4d0d-a3b2-1846e47d6446 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:38.084957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 65/127] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:37 2022] GET /identity/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:38.085392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3718c-ecac-4d0d-a3b2-1846e47d6446 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:38.135021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:38.140855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d65eed-41b2-49a7-ba74-f4a9a17e3d26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:38.142943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d65eed-41b2-49a7-ba74-f4a9a17e3d26 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:38.217903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3718c-ecac-4d0d-a3b2-1846e47d6446 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:38.255399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90b98a-c65f-460d-82c4-dc89f8954530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:38.256031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90b98a-c65f-460d-82c4-dc89f8954530 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:38.257425 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90b98a-c65f-460d-82c4-dc89f8954530 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:38.257704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90b98a-c65f-460d-82c4-dc89f8954530 None admin] PATH_INFO: `/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:38.259421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:38.259857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90b98a-c65f-460d-82c4-dc89f8954530 None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf9cae9ba447d4851f472b6e46e3fd)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:38.260937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90b98a-c65f-460d-82c4-dc89f8954530 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:38.262733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 66/128] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:38 2022] GET /identity/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:38.265499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90b98a-c65f-460d-82c4-dc89f8954530 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:38.273709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 63/129] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:38.274549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d65eed-41b2-49a7-ba74-f4a9a17e3d26 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:38.382658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d65eed-41b2-49a7-ba74-f4a9a17e3d26 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:38.401213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90b98a-c65f-460d-82c4-dc89f8954530 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:38.411510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:38.423336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0bfce-77a8-4e03-b4c0-1b9b4bdbd2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:38.424399 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0bfce-77a8-4e03-b4c0-1b9b4bdbd2b1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:38.424781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0bfce-77a8-4e03-b4c0-1b9b4bdbd2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:38.425153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0bfce-77a8-4e03-b4c0-1b9b4bdbd2b1 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:38.426406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0bfce-77a8-4e03-b4c0-1b9b4bdbd2b1 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:38.427956 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0bfce-77a8-4e03-b4c0-1b9b4bdbd2b1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:38.429915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 64/130] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:38 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:38.435370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0bfce-77a8-4e03-b4c0-1b9b4bdbd2b1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:38.454030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6234a-b28b-4e82-b2fe-b66a3f471df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:38.455079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6234a-b28b-4e82-b2fe-b66a3f471df8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:38.455819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6234a-b28b-4e82-b2fe-b66a3f471df8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:38.456283 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6234a-b28b-4e82-b2fe-b66a3f471df8 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:38.457733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6234a-b28b-4e82-b2fe-b66a3f471df8 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:38.460027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6234a-b28b-4e82-b2fe-b66a3f471df8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:38.462414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 67/131] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:38 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:38.463791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6234a-b28b-4e82-b2fe-b66a3f471df8 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:38.502545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:38.550584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0bfce-77a8-4e03-b4c0-1b9b4bdbd2b1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:38.595173 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fcb2f-38f7-43a9-bb84-3af554e0e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:38.596350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fcb2f-38f7-43a9-bb84-3af554e0e25c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:38.596797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fcb2f-38f7-43a9-bb84-3af554e0e25c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:38.597217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fcb2f-38f7-43a9-bb84-3af554e0e25c None admin] PATH_INFO: `/v3/projects/883884da6bf64da0afd09d6d4956d7f2` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:38.597987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6234a-b28b-4e82-b2fe-b66a3f471df8 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:38.598688 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fcb2f-38f7-43a9-bb84-3af554e0e25c None admin] RBAC: Authorizing `identity:get_project(project_id=883884da6bf64da0afd09d6d4956d7f2)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:38.601497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fcb2f-38f7-43a9-bb84-3af554e0e25c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:38.603317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 65/132] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:38 2022] GET /identity/v3/projects/883884da6bf64da0afd09d6d4956d7f2 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:38.604269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fcb2f-38f7-43a9-bb84-3af554e0e25c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:38.638646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25896d-f527-4131-9534-15a720598223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:38.639378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25896d-f527-4131-9534-15a720598223 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:38.639756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25896d-f527-4131-9534-15a720598223 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:38.640129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25896d-f527-4131-9534-15a720598223 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:38.641580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25896d-f527-4131-9534-15a720598223 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:38.643241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25896d-f527-4131-9534-15a720598223 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:38.645225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 68/133] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:38 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:38.646518 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25896d-f527-4131-9534-15a720598223 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:38.698883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:38.752750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fcb2f-38f7-43a9-bb84-3af554e0e25c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:38.772103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25896d-f527-4131-9534-15a720598223 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:38.789275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:38.800291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb23bc-9525-4b88-bd72-643c7c24a140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:38.801733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb23bc-9525-4b88-bd72-643c7c24a140 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:38.802223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb23bc-9525-4b88-bd72-643c7c24a140 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:38.802909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb23bc-9525-4b88-bd72-643c7c24a140 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/users/c0bf9cae9ba447d4851f472b6e46e3fd/roles/4bb04c67143448539c5af8d8bdd621f9` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:38.806338 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb23bc-9525-4b88-bd72-643c7c24a140 None admin] RBAC: Authorizing `identity:create_grant(project_id=973ebf7f6f414136a6ae5a7d1a1a919c, user_id=c0bf9cae9ba447d4851f472b6e46e3fd, role_id=4bb04c67143448539c5af8d8bdd621f9)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:38.808705 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb23bc-9525-4b88-bd72-643c7c24a140 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:38.833109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989630d3-20fb-42ee-a997-5f7b4294c9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:38.833794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989630d3-20fb-42ee-a997-5f7b4294c9da None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:38.839347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989630d3-20fb-42ee-a997-5f7b4294c9da None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:38.839347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989630d3-20fb-42ee-a997-5f7b4294c9da None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/users/374ef1a08ea74e7cb0bf480e25c8cdd4/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:38.839347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989630d3-20fb-42ee-a997-5f7b4294c9da None admin] RBAC: Authorizing `identity:create_grant(project_id=973ebf7f6f414136a6ae5a7d1a1a919c, user_id=374ef1a08ea74e7cb0bf480e25c8cdd4, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:38.839347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989630d3-20fb-42ee-a997-5f7b4294c9da None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:38.912161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 66/134] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:39:38 2022] PUT /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/users/c0bf9cae9ba447d4851f472b6e46e3fd/roles/4bb04c67143448539c5af8d8bdd621f9 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:39:38.913538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb23bc-9525-4b88-bd72-643c7c24a140 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:38.940704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 69/135] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:39:38 2022] PUT /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/users/374ef1a08ea74e7cb0bf480e25c8cdd4/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:39:38.942042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989630d3-20fb-42ee-a997-5f7b4294c9da None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:39.086860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989630d3-20fb-42ee-a997-5f7b4294c9da None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:39.089203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:39.136401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675ee79-5f02-4018-8910-0311e3955101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:39.137337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675ee79-5f02-4018-8910-0311e3955101 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:39.138394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675ee79-5f02-4018-8910-0311e3955101 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:39.138784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675ee79-5f02-4018-8910-0311e3955101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:39.139835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675ee79-5f02-4018-8910-0311e3955101 None admin] RBAC: Adding query filter params (role.id=7ae225f5f02c4cefaeb687c582acfcca, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c, user.id=d972b329d9a74fdaac32d9cf48c2e9dc) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:39.140428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675ee79-5f02-4018-8910-0311e3955101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:39.142043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675ee79-5f02-4018-8910-0311e3955101 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:39.142478 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb23bc-9525-4b88-bd72-643c7c24a140 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:39.161751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 70/136] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:38 2022] GET /identity/v3/role_assignments?user.id=d972b329d9a74fdaac32d9cf48c2e9dc&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=7ae225f5f02c4cefaeb687c582acfcca => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:39.161751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0525c-1e55-4711-a88b-c85cfbfe7f10 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:39.161751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0525c-1e55-4711-a88b-c85cfbfe7f10 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:39.161751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0525c-1e55-4711-a88b-c85cfbfe7f10 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:39.161751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 71/137] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:39.172198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b7cc4-7ad4-4dd0-8219-9fbc6cfdfa41 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:39.172198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b7cc4-7ad4-4dd0-8219-9fbc6cfdfa41 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:39.172198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b7cc4-7ad4-4dd0-8219-9fbc6cfdfa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:39.203051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-389b7cc4-7ad4-4dd0-8219-9fbc6cfdfa41 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:39.205260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-389b7cc4-7ad4-4dd0-8219-9fbc6cfdfa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:39.241099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c9015-1ee6-4cd2-895f-c4f9318411cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:39.241850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c9015-1ee6-4cd2-895f-c4f9318411cd None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:39.242119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c9015-1ee6-4cd2-895f-c4f9318411cd None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:39.242393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c9015-1ee6-4cd2-895f-c4f9318411cd None admin] PATH_INFO: `/v3/projects/883884da6bf64da0afd09d6d4956d7f2/users/c0bf9cae9ba447d4851f472b6e46e3fd/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:39.248593 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c9015-1ee6-4cd2-895f-c4f9318411cd None admin] RBAC: Authorizing `identity:create_grant(project_id=883884da6bf64da0afd09d6d4956d7f2, user_id=c0bf9cae9ba447d4851f472b6e46e3fd, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:39.250083 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c9015-1ee6-4cd2-895f-c4f9318411cd None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:39.298632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 72/138] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:39.327318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 67/139] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:39:38 2022] PUT /identity/v3/projects/883884da6bf64da0afd09d6d4956d7f2/users/c0bf9cae9ba447d4851f472b6e46e3fd/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:39:39.629634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b7cc4-7ad4-4dd0-8219-9fbc6cfdfa41 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:39.752614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:39.761637 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b7cc4-7ad4-4dd0-8219-9fbc6cfdfa41 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:39.805338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bcd14-22b2-4024-8cda-ba1b245081b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:39.806152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bcd14-22b2-4024-8cda-ba1b245081b7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:39.806543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bcd14-22b2-4024-8cda-ba1b245081b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:39.806926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bcd14-22b2-4024-8cda-ba1b245081b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:39.825463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bcd14-22b2-4024-8cda-ba1b245081b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-521bcd14-22b2-4024-8cda-ba1b245081b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:39.832439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:39.833551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:39.834561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:39.836454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 73/140] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:39.845339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c9015-1ee6-4cd2-895f-c4f9318411cd None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:39.937385 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:40.010568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c9015-1ee6-4cd2-895f-c4f9318411cd None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:40.024704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0c03-4af6-464c-9343-3098ad87c068 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:40.025318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0c03-4af6-464c-9343-3098ad87c068 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:40.025860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0c03-4af6-464c-9343-3098ad87c068 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:40.028343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 74/141] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:40.034026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38555cda-7ab3-4e17-afd6-858b063f3012 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:40.035182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38555cda-7ab3-4e17-afd6-858b063f3012 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:40.035669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38555cda-7ab3-4e17-afd6-858b063f3012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:40.069051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-38555cda-7ab3-4e17-afd6-858b063f3012 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:40.070882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-38555cda-7ab3-4e17-afd6-858b063f3012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:40.088413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02034703-7577-496c-9b85-63492e135020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:40.089050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02034703-7577-496c-9b85-63492e135020 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:40.089339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02034703-7577-496c-9b85-63492e135020 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:40.089619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02034703-7577-496c-9b85-63492e135020 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:40.090740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02034703-7577-496c-9b85-63492e135020 None admin] RBAC: Adding query filter params (name=member) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:40.090850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02034703-7577-496c-9b85-63492e135020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:40.092263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02034703-7577-496c-9b85-63492e135020 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:40.110680 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 68/142] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:40.120196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02034703-7577-496c-9b85-63492e135020 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:40.199116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 75/143] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:39:40.300108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02034703-7577-496c-9b85-63492e135020 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:40.319194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:40.358465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1566b-a23e-4202-991a-886f01a34aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:40.359953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1566b-a23e-4202-991a-886f01a34aed None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:40.360465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1566b-a23e-4202-991a-886f01a34aed None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:40.360946 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1566b-a23e-4202-991a-886f01a34aed None admin] PATH_INFO: `/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:40.362410 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1566b-a23e-4202-991a-886f01a34aed None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf9cae9ba447d4851f472b6e46e3fd)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:40.364110 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1566b-a23e-4202-991a-886f01a34aed None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:40.366028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 69/144] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:40 2022] GET /identity/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:40.371687 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38555cda-7ab3-4e17-afd6-858b063f3012 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:40.517621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:40.541212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38555cda-7ab3-4e17-afd6-858b063f3012 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:40.602652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8acb77-8807-44f2-80e1-440c0093269d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:40.603550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8acb77-8807-44f2-80e1-440c0093269d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:40.603834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8acb77-8807-44f2-80e1-440c0093269d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:40.604109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8acb77-8807-44f2-80e1-440c0093269d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:40.711979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8acb77-8807-44f2-80e1-440c0093269d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:40.713505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8acb77-8807-44f2-80e1-440c0093269d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:40.773257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1566b-a23e-4202-991a-886f01a34aed None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:40.910576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:40.934453 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 76/145] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:40.940895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8acb77-8807-44f2-80e1-440c0093269d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:40.960891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1566b-a23e-4202-991a-886f01a34aed None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:41.009667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825717b3-3f4f-4ec2-95c1-5456765ee1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:41.010481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825717b3-3f4f-4ec2-95c1-5456765ee1e9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:41.010863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825717b3-3f4f-4ec2-95c1-5456765ee1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:41.011236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825717b3-3f4f-4ec2-95c1-5456765ee1e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:41.012855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825717b3-3f4f-4ec2-95c1-5456765ee1e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:41.014845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825717b3-3f4f-4ec2-95c1-5456765ee1e9 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:41.016659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 70/146] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:41.023203 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825717b3-3f4f-4ec2-95c1-5456765ee1e9 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:41.098825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:41.200614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8acb77-8807-44f2-80e1-440c0093269d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:41.234661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825717b3-3f4f-4ec2-95c1-5456765ee1e9 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:41.287871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487fa56-1261-46c9-8107-9d70c9878ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:41.288527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487fa56-1261-46c9-8107-9d70c9878ff2 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:41.288807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487fa56-1261-46c9-8107-9d70c9878ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:41.289085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487fa56-1261-46c9-8107-9d70c9878ff2 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:41.289441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f10c2d-29a0-45e4-a980-769d86fffb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:41.289748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487fa56-1261-46c9-8107-9d70c9878ff2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:41.290088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f10c2d-29a0-45e4-a980-769d86fffb56 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:41.290366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f10c2d-29a0-45e4-a980-769d86fffb56 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:41.290640 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f10c2d-29a0-45e4-a980-769d86fffb56 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:41.291196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487fa56-1261-46c9-8107-9d70c9878ff2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:41.296883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f10c2d-29a0-45e4-a980-769d86fffb56 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:41.298354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f10c2d-29a0-45e4-a980-769d86fffb56 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:41.300383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 77/147] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:40 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:41.305991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f10c2d-29a0-45e4-a980-769d86fffb56 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:41.395907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 71/148] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:39:41 2022] POST /identity/v3/users => generated 283 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:39:41.404254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:41.465676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f10c2d-29a0-45e4-a980-769d86fffb56 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:41.503992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87409aa5-5083-4715-b5f7-5b1e7e8f0677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:41.504653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87409aa5-5083-4715-b5f7-5b1e7e8f0677 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:41.504980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87409aa5-5083-4715-b5f7-5b1e7e8f0677 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:41.505255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87409aa5-5083-4715-b5f7-5b1e7e8f0677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:41.505990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87409aa5-5083-4715-b5f7-5b1e7e8f0677 None admin] RBAC: Adding query filter params (role.id=3f0c34a1f93e49adb1757f315333cc48, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c, user.id=c0bf9cae9ba447d4851f472b6e46e3fd) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:41.506297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87409aa5-5083-4715-b5f7-5b1e7e8f0677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:41.507997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87409aa5-5083-4715-b5f7-5b1e7e8f0677 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:41.515764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 78/149] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:41 2022] GET /identity/v3/role_assignments?user.id=c0bf9cae9ba447d4851f472b6e46e3fd&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:41.947893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd1e9f-9681-47ae-9b9f-aafb7ed43891 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:41.948296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd1e9f-9681-47ae-9b9f-aafb7ed43891 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:41.948490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd1e9f-9681-47ae-9b9f-aafb7ed43891 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:41.950044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 72/150] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:41.957350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec3ace-7615-4873-97bd-5f07100d514d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:41.957350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec3ace-7615-4873-97bd-5f07100d514d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:41.957350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec3ace-7615-4873-97bd-5f07100d514d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:41.986965 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b3ec3ace-7615-4873-97bd-5f07100d514d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:41.988546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b3ec3ace-7615-4873-97bd-5f07100d514d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:42.091219 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 79/151] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:39:42.467583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec8feb-b122-4850-a9e7-e659462e39c6 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:42.468082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec8feb-b122-4850-a9e7-e659462e39c6 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:42.468082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec8feb-b122-4850-a9e7-e659462e39c6 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:42.469718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 73/152] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:42.478292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bca84b-e744-473f-a87d-27ddb3d49bc8 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:42.478792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bca84b-e744-473f-a87d-27ddb3d49bc8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:42.478863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bca84b-e744-473f-a87d-27ddb3d49bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:42.506888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-81bca84b-e744-473f-a87d-27ddb3d49bc8 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:42.508459 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-81bca84b-e744-473f-a87d-27ddb3d49bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:42.572840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:42.597947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec8feb-b122-4850-a9e7-e659462e39c6 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:42.606590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 80/153] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:42.615569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50998f4-8cd6-4803-b162-12558e0be9bf None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:42.616086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50998f4-8cd6-4803-b162-12558e0be9bf None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:42.616535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50998f4-8cd6-4803-b162-12558e0be9bf None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:42.618949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 81/154] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:42.628554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c0942-7293-4d24-9ab5-5ff20444a80a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:42.629089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c0942-7293-4d24-9ab5-5ff20444a80a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:42.629500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c0942-7293-4d24-9ab5-5ff20444a80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:42.651946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-026c0942-7293-4d24-9ab5-5ff20444a80a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:42.654694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-026c0942-7293-4d24-9ab5-5ff20444a80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:42.739264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 82/155] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:42.870255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6986b28-3ba2-4cae-82f4-8cda359840f1 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:42.870794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6986b28-3ba2-4cae-82f4-8cda359840f1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:42.871209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6986b28-3ba2-4cae-82f4-8cda359840f1 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:42.873602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 83/156] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:39:42.881297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a054b0-89bc-4865-ba1c-44345d822213 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:42.881776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a054b0-89bc-4865-ba1c-44345d822213 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:42.882225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a054b0-89bc-4865-ba1c-44345d822213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:42.894068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec8feb-b122-4850-a9e7-e659462e39c6 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:42.896929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:42.911087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-69a054b0-89bc-4865-ba1c-44345d822213 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:42.912666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-69a054b0-89bc-4865-ba1c-44345d822213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:42.982790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741c5f7-1ccb-48a0-abb7-41d673eae83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:42.983480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741c5f7-1ccb-48a0-abb7-41d673eae83a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:42.983763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741c5f7-1ccb-48a0-abb7-41d673eae83a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:42.984040 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741c5f7-1ccb-48a0-abb7-41d673eae83a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:42.987483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 84/157] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:43.009315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741c5f7-1ccb-48a0-abb7-41d673eae83a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-3741c5f7-1ccb-48a0-abb7-41d673eae83a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:43.014944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:43.016081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:43.017417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 74/158] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:39:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:43.025561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a054b0-89bc-4865-ba1c-44345d822213 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:43.082297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:43.149858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a054b0-89bc-4865-ba1c-44345d822213 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:43.192944 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030cb4a-708a-44e1-8897-a9b8c575f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:43.193580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030cb4a-708a-44e1-8897-a9b8c575f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:43.193863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030cb4a-708a-44e1-8897-a9b8c575f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:43.194140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030cb4a-708a-44e1-8897-a9b8c575f8f0 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:43.194826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030cb4a-708a-44e1-8897-a9b8c575f8f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:43.195137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030cb4a-708a-44e1-8897-a9b8c575f8f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:43.196590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030cb4a-708a-44e1-8897-a9b8c575f8f0 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:43.203744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741c5f7-1ccb-48a0-abb7-41d673eae83a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:43.206650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 85/159] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:43.212641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030cb4a-708a-44e1-8897-a9b8c575f8f0 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:43.329158 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:43.332633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030cb4a-708a-44e1-8897-a9b8c575f8f0 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:43.380816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344046d-6257-44a3-9a6c-ee3fffc66c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:43.381469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344046d-6257-44a3-9a6c-ee3fffc66c74 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:43.381748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344046d-6257-44a3-9a6c-ee3fffc66c74 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:43.382021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344046d-6257-44a3-9a6c-ee3fffc66c74 None admin] PATH_INFO: `/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:43.385028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344046d-6257-44a3-9a6c-ee3fffc66c74 None admin] RBAC: Authorizing `identity:get_user(user_id=374ef1a08ea74e7cb0bf480e25c8cdd4)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:43.386481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344046d-6257-44a3-9a6c-ee3fffc66c74 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:43.388150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 86/160] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:43 2022] GET /identity/v3/users/374ef1a08ea74e7cb0bf480e25c8cdd4 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:43.389049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344046d-6257-44a3-9a6c-ee3fffc66c74 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:43.443055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741c5f7-1ccb-48a0-abb7-41d673eae83a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:43.503891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344046d-6257-44a3-9a6c-ee3fffc66c74 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:43.540380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6553d0-979b-4a42-b0f0-9fcc89e6aee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:43.541046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6553d0-979b-4a42-b0f0-9fcc89e6aee1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:43.541333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6553d0-979b-4a42-b0f0-9fcc89e6aee1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:43.541614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6553d0-979b-4a42-b0f0-9fcc89e6aee1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:43.545666 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5c14c-31c1-4ac1-8063-b0a5b245e133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:43.546322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c14c-31c1-4ac1-8063-b0a5b245e133 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:43.546584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c14c-31c1-4ac1-8063-b0a5b245e133 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:43.546855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5c14c-31c1-4ac1-8063-b0a5b245e133 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:43.553782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6553d0-979b-4a42-b0f0-9fcc89e6aee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-4d6553d0-979b-4a42-b0f0-9fcc89e6aee1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:43.560033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:43.561033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:43.561835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 87/161] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:39:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:43.562488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6553d0-979b-4a42-b0f0-9fcc89e6aee1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:43.567141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5c14c-31c1-4ac1-8063-b0a5b245e133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-a1d5c14c-31c1-4ac1-8063-b0a5b245e133 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:43.578234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:43.580514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 75/162] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:43.582446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5c14c-31c1-4ac1-8063-b0a5b245e133 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:43.604550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:43.639198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:43.669702 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6553d0-979b-4a42-b0f0-9fcc89e6aee1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:43.709134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c473f8-c70a-4c79-b278-4a1bd7e944b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:43.709752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c473f8-c70a-4c79-b278-4a1bd7e944b7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:43.710035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c473f8-c70a-4c79-b278-4a1bd7e944b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:43.710310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c473f8-c70a-4c79-b278-4a1bd7e944b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:43.790296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c473f8-c70a-4c79-b278-4a1bd7e944b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:43.791815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c473f8-c70a-4c79-b278-4a1bd7e944b7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:43.828942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:43.841399 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5c14c-31c1-4ac1-8063-b0a5b245e133 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:43.911836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 88/163] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:43.912703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c473f8-c70a-4c79-b278-4a1bd7e944b7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:43.937029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6ac97-3983-4311-9219-578ab489e0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:43.938133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6ac97-3983-4311-9219-578ab489e0b7 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:43.938646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6ac97-3983-4311-9219-578ab489e0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:43.948765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6ac97-3983-4311-9219-578ab489e0b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:43.974316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6ac97-3983-4311-9219-578ab489e0b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-47e6ac97-3983-4311-9219-578ab489e0b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:43.994398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:43.995592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:43.996446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:43.998315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 76/164] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:43.999361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6ac97-3983-4311-9219-578ab489e0b7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:44.019058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c473f8-c70a-4c79-b278-4a1bd7e944b7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:44.048869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:44.061162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3df85-873a-4183-a5bb-59a3512f5d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:44.061635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3df85-873a-4183-a5bb-59a3512f5d44 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:44.062390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3df85-873a-4183-a5bb-59a3512f5d44 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:44.062669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3df85-873a-4183-a5bb-59a3512f5d44 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:44.063348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3df85-873a-4183-a5bb-59a3512f5d44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:44.063653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3df85-873a-4183-a5bb-59a3512f5d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:44.065481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3df85-873a-4183-a5bb-59a3512f5d44 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:44.076286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 89/165] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:39:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:44.077175 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3df85-873a-4183-a5bb-59a3512f5d44 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:44.193188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3df85-873a-4183-a5bb-59a3512f5d44 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:44.230744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0609-2cea-4d42-b822-8f0a0c7871c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:44.231404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0609-2cea-4d42-b822-8f0a0c7871c0 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:44.231719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0609-2cea-4d42-b822-8f0a0c7871c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:44.232274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0609-2cea-4d42-b822-8f0a0c7871c0 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:44.233448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0609-2cea-4d42-b822-8f0a0c7871c0 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:44.234897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0609-2cea-4d42-b822-8f0a0c7871c0 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:44.236768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 90/166] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:44 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:44.237541 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0609-2cea-4d42-b822-8f0a0c7871c0 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:44.271365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:44.304985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:44.321678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6ac97-3983-4311-9219-578ab489e0b7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:44.351157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0609-2cea-4d42-b822-8f0a0c7871c0 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:44.389535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1521b-681a-4caa-a3a4-441ca4024bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:44.390154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1521b-681a-4caa-a3a4-441ca4024bec None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:44.390439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1521b-681a-4caa-a3a4-441ca4024bec None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:44.390689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1521b-681a-4caa-a3a4-441ca4024bec None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:44.391373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1521b-681a-4caa-a3a4-441ca4024bec None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:44.391683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1521b-681a-4caa-a3a4-441ca4024bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:44.393481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1521b-681a-4caa-a3a4-441ca4024bec None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:44.402996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 91/167] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:44.404560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1521b-681a-4caa-a3a4-441ca4024bec None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:44.440924 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f79f1-593b-485d-b697-d7fdd47ade69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:44.441553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f79f1-593b-485d-b697-d7fdd47ade69 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:44.441841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f79f1-593b-485d-b697-d7fdd47ade69 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:44.442106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f79f1-593b-485d-b697-d7fdd47ade69 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:44.442793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f79f1-593b-485d-b697-d7fdd47ade69 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:44.443102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f79f1-593b-485d-b697-d7fdd47ade69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:44.444610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f79f1-593b-485d-b697-d7fdd47ade69 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:44.461498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 77/168] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:44.462151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f79f1-593b-485d-b697-d7fdd47ade69 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:44.587441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1521b-681a-4caa-a3a4-441ca4024bec None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:44.619959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:44.652386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89306f9-bacc-4da5-9a24-471e92d2e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:44.653020 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89306f9-bacc-4da5-9a24-471e92d2e60b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:44.653423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89306f9-bacc-4da5-9a24-471e92d2e60b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:44.653699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89306f9-bacc-4da5-9a24-471e92d2e60b None admin] PATH_INFO: `/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:44.656044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89306f9-bacc-4da5-9a24-471e92d2e60b None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf9cae9ba447d4851f472b6e46e3fd)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:44.657515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89306f9-bacc-4da5-9a24-471e92d2e60b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:44.660236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 92/169] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:44 2022] GET /identity/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:44.661813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89306f9-bacc-4da5-9a24-471e92d2e60b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:44.673148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f79f1-593b-485d-b697-d7fdd47ade69 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:44.741750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa54991-97f4-445b-8200-eb6dc9e96652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:44.742251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54991-97f4-445b-8200-eb6dc9e96652 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:44.742521 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54991-97f4-445b-8200-eb6dc9e96652 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:44.742802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54991-97f4-445b-8200-eb6dc9e96652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:44.743510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa54991-97f4-445b-8200-eb6dc9e96652 None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c, user.id=374ef1a08ea74e7cb0bf480e25c8cdd4) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:44.743813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa54991-97f4-445b-8200-eb6dc9e96652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:44.745278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa54991-97f4-445b-8200-eb6dc9e96652 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:44.759141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 78/170] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:44 2022] GET /identity/v3/role_assignments?user.id=374ef1a08ea74e7cb0bf480e25c8cdd4&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:44.765152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa54991-97f4-445b-8200-eb6dc9e96652 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:44.788876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:44.932202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa54991-97f4-445b-8200-eb6dc9e96652 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:44.996829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd1f8f-cc11-4b02-81c8-5a857a020e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:44.998154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1f8f-cc11-4b02-81c8-5a857a020e6e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:44.998444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1f8f-cc11-4b02-81c8-5a857a020e6e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:44.998720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1f8f-cc11-4b02-81c8-5a857a020e6e None admin] PATH_INFO: `/v3/users/d972b329d9a74fdaac32d9cf48c2e9dc` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:45.001099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd1f8f-cc11-4b02-81c8-5a857a020e6e None admin] RBAC: Authorizing `identity:get_user(user_id=d972b329d9a74fdaac32d9cf48c2e9dc)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:45.002620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd1f8f-cc11-4b02-81c8-5a857a020e6e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:45.004389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 79/171] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:44 2022] GET /identity/v3/users/d972b329d9a74fdaac32d9cf48c2e9dc => generated 289 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:45.005198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd1f8f-cc11-4b02-81c8-5a857a020e6e None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:45.006450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89306f9-bacc-4da5-9a24-471e92d2e60b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:45.084270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3284dc1-9e81-447b-aabe-b177999983dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:45.084931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3284dc1-9e81-447b-aabe-b177999983dc None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:45.085201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3284dc1-9e81-447b-aabe-b177999983dc None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:45.085478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3284dc1-9e81-447b-aabe-b177999983dc None admin] PATH_INFO: `/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:45.090325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3284dc1-9e81-447b-aabe-b177999983dc None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf9cae9ba447d4851f472b6e46e3fd)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:45.092513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3284dc1-9e81-447b-aabe-b177999983dc None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:45.100762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 93/172] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:44 2022] GET /identity/v3/users/c0bf9cae9ba447d4851f472b6e46e3fd => generated 275 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:45.101751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3284dc1-9e81-447b-aabe-b177999983dc None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:45.201024 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:45.242091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd1f8f-cc11-4b02-81c8-5a857a020e6e None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:45.246782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:45.302756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00631ffc-8604-4db3-8787-dcf09926c810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:45.303592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00631ffc-8604-4db3-8787-dcf09926c810 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:45.304103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00631ffc-8604-4db3-8787-dcf09926c810 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:45.305610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00631ffc-8604-4db3-8787-dcf09926c810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:45.308309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3284dc1-9e81-447b-aabe-b177999983dc None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:45.350787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180cfc5-ca32-4da3-b996-58492ef3da30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:45.351598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180cfc5-ca32-4da3-b996-58492ef3da30 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:45.352332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180cfc5-ca32-4da3-b996-58492ef3da30 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:45.352332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180cfc5-ca32-4da3-b996-58492ef3da30 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:45.353570 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180cfc5-ca32-4da3-b996-58492ef3da30 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:45.355126 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180cfc5-ca32-4da3-b996-58492ef3da30 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:45.357605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 94/173] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:45 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:45.358544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180cfc5-ca32-4da3-b996-58492ef3da30 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:45.387136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00631ffc-8604-4db3-8787-dcf09926c810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:45.389420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00631ffc-8604-4db3-8787-dcf09926c810 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:45.479249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:45.482631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180cfc5-ca32-4da3-b996-58492ef3da30 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:45.507703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 80/174] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:45.509447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0a4aa-9363-42f1-a2f3-148a384f8795 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:45.509911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0a4aa-9363-42f1-a2f3-148a384f8795 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:45.510594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0a4aa-9363-42f1-a2f3-148a384f8795 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:45.516922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 81/175] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:45.517688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:45.519516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0a4aa-9363-42f1-a2f3-148a384f8795 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:45.527417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4fb15-c314-44c5-bd50-13cfcd88518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:45.528036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4fb15-c314-44c5-bd50-13cfcd88518e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:45.529748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4fb15-c314-44c5-bd50-13cfcd88518e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:45.529748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4fb15-c314-44c5-bd50-13cfcd88518e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:45.623796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4fb15-c314-44c5-bd50-13cfcd88518e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:45.625492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4fb15-c314-44c5-bd50-13cfcd88518e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:45.658536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0a4aa-9363-42f1-a2f3-148a384f8795 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:45.710255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca3514-f1ea-44a5-be99-fec658527748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:45.712212 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca3514-f1ea-44a5-be99-fec658527748 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:45.713050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca3514-f1ea-44a5-be99-fec658527748 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:45.713535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca3514-f1ea-44a5-be99-fec658527748 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/users/d972b329d9a74fdaac32d9cf48c2e9dc/roles/7ae225f5f02c4cefaeb687c582acfcca` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:45.717905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca3514-f1ea-44a5-be99-fec658527748 None admin] RBAC: Authorizing `identity:create_grant(project_id=973ebf7f6f414136a6ae5a7d1a1a919c, user_id=d972b329d9a74fdaac32d9cf48c2e9dc, role_id=7ae225f5f02c4cefaeb687c582acfcca)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:45.719778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca3514-f1ea-44a5-be99-fec658527748 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:45.750755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 95/176] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:45.755738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4fb15-c314-44c5-bd50-13cfcd88518e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:45.781887 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:45.804039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 82/177] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:39:45 2022] PUT /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/users/d972b329d9a74fdaac32d9cf48c2e9dc/roles/7ae225f5f02c4cefaeb687c582acfcca => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:39:45.805827 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70e210-687a-42c5-8e35-60eb5bebcbcf None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:45.806336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70e210-687a-42c5-8e35-60eb5bebcbcf None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:45.807510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70e210-687a-42c5-8e35-60eb5bebcbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:45.851277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-3a70e210-687a-42c5-8e35-60eb5bebcbcf None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:45.853847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3a70e210-687a-42c5-8e35-60eb5bebcbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:45.902209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:45.912658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4fb15-c314-44c5-bd50-13cfcd88518e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:45.951537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 83/178] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:45.958450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70e210-687a-42c5-8e35-60eb5bebcbcf None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:45.964526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4151a-2519-4e89-8c14-f84335516135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:45.965555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4151a-2519-4e89-8c14-f84335516135 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:45.966072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4151a-2519-4e89-8c14-f84335516135 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:45.967726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4151a-2519-4e89-8c14-f84335516135 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:45.971498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4151a-2519-4e89-8c14-f84335516135 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:45.973372 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4151a-2519-4e89-8c14-f84335516135 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:45.977707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 96/179] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:45 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:45.980299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8060a6-fb9d-4926-b41b-d67d0a5c637b None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:45.982611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8060a6-fb9d-4926-b41b-d67d0a5c637b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:45.983088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8060a6-fb9d-4926-b41b-d67d0a5c637b None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:45.985162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 97/180] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:45.988100 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8060a6-fb9d-4926-b41b-d67d0a5c637b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:46.133417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:46.147470 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8060a6-fb9d-4926-b41b-d67d0a5c637b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:46.161477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70e210-687a-42c5-8e35-60eb5bebcbcf None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:46.194553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881a40b-f025-4842-99fc-fda03ee67a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:46.195661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881a40b-f025-4842-99fc-fda03ee67a47 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:46.196122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881a40b-f025-4842-99fc-fda03ee67a47 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:46.196992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881a40b-f025-4842-99fc-fda03ee67a47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:46.197917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881a40b-f025-4842-99fc-fda03ee67a47 None admin] RBAC: Adding query filter params (role.id=4bb04c67143448539c5af8d8bdd621f9, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c, user.id=c0bf9cae9ba447d4851f472b6e46e3fd) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:46.198417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881a40b-f025-4842-99fc-fda03ee67a47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:46.201103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881a40b-f025-4842-99fc-fda03ee67a47 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:46.208588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 98/181] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:45 2022] GET /identity/v3/role_assignments?user.id=c0bf9cae9ba447d4851f472b6e46e3fd&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=4bb04c67143448539c5af8d8bdd621f9 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:46.210430 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322dc2e-c004-43aa-abea-58b4d67d4495 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:46.211043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322dc2e-c004-43aa-abea-58b4d67d4495 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:46.211495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322dc2e-c004-43aa-abea-58b4d67d4495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:46.237595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2322dc2e-c004-43aa-abea-58b4d67d4495 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:46.239135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2322dc2e-c004-43aa-abea-58b4d67d4495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:46.240668 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60259b7d-6c49-4972-93a4-1c8bfc1e7107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:46.241420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60259b7d-6c49-4972-93a4-1c8bfc1e7107 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:46.241786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60259b7d-6c49-4972-93a4-1c8bfc1e7107 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:46.242164 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60259b7d-6c49-4972-93a4-1c8bfc1e7107 None admin] PATH_INFO: `/v3/projects/883884da6bf64da0afd09d6d4956d7f2` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:46.245091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60259b7d-6c49-4972-93a4-1c8bfc1e7107 None admin] RBAC: Authorizing `identity:get_project(project_id=883884da6bf64da0afd09d6d4956d7f2)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:46.246678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60259b7d-6c49-4972-93a4-1c8bfc1e7107 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:46.254206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 84/182] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:45 2022] GET /identity/v3/projects/883884da6bf64da0afd09d6d4956d7f2 => generated 312 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:46.269710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60259b7d-6c49-4972-93a4-1c8bfc1e7107 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:46.321157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:46.333789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 99/183] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:46.335525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa6aa1-2e32-47f8-bd9a-2ccf83a5caa1 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:46.336806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa6aa1-2e32-47f8-bd9a-2ccf83a5caa1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:46.336806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa6aa1-2e32-47f8-bd9a-2ccf83a5caa1 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:46.338050 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 100/184] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:46.347799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01624e93-2206-4f94-bb36-93fcd8be558a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:46.349256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01624e93-2206-4f94-bb36-93fcd8be558a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:46.349713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01624e93-2206-4f94-bb36-93fcd8be558a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:46.391612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-01624e93-2206-4f94-bb36-93fcd8be558a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:46.393319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-01624e93-2206-4f94-bb36-93fcd8be558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:46.476304 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60259b7d-6c49-4972-93a4-1c8bfc1e7107 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:46.531760 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 101/185] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:46.533611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01624e93-2206-4f94-bb36-93fcd8be558a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:46.538801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87f1ad-b991-4d17-8057-ba07fa633eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:46.539772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87f1ad-b991-4d17-8057-ba07fa633eaf None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:46.540276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87f1ad-b991-4d17-8057-ba07fa633eaf None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:46.540814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87f1ad-b991-4d17-8057-ba07fa633eaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:46.541885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87f1ad-b991-4d17-8057-ba07fa633eaf None admin] RBAC: Adding query filter params (role.id=3f0c34a1f93e49adb1757f315333cc48, scope.project.id=883884da6bf64da0afd09d6d4956d7f2, user.id=c0bf9cae9ba447d4851f472b6e46e3fd) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:46.542405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87f1ad-b991-4d17-8057-ba07fa633eaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:46.544334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87f1ad-b991-4d17-8057-ba07fa633eaf None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:46.564793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 85/186] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:46 2022] GET /identity/v3/role_assignments?user.id=c0bf9cae9ba447d4851f472b6e46e3fd&scope.project.id=883884da6bf64da0afd09d6d4956d7f2&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:46.573111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:46.620962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87f1ad-b991-4d17-8057-ba07fa633eaf None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:46.697737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01624e93-2206-4f94-bb36-93fcd8be558a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:46.739846 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff28ff-3364-4240-a010-526487742743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:46.740994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff28ff-3364-4240-a010-526487742743 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:46.741289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff28ff-3364-4240-a010-526487742743 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:46.741570 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff28ff-3364-4240-a010-526487742743 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:46.757213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff28ff-3364-4240-a010-526487742743 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-03ff28ff-3364-4240-a010-526487742743 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:46.763476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:46.764445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:46.765228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 102/187] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:46.773414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff28ff-3364-4240-a010-526487742743 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:46.783007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:46.792854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87f1ad-b991-4d17-8057-ba07fa633eaf None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:46.933095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13f54-cf45-42a1-82bb-2e4502e551f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:46.933769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13f54-cf45-42a1-82bb-2e4502e551f3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:46.934058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13f54-cf45-42a1-82bb-2e4502e551f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:46.934370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13f54-cf45-42a1-82bb-2e4502e551f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:46.935706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d13f54-cf45-42a1-82bb-2e4502e551f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:46.940221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d13f54-cf45-42a1-82bb-2e4502e551f3 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:46.940221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 86/188] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:46.949270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13f54-cf45-42a1-82bb-2e4502e551f3 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:46.960723 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:47.005441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff28ff-3364-4240-a010-526487742743 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:47.049099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b524b-3a52-402e-84ae-e3a40b4d04a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:47.049854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b524b-3a52-402e-84ae-e3a40b4d04a3 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:47.050111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b524b-3a52-402e-84ae-e3a40b4d04a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:47.050718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b524b-3a52-402e-84ae-e3a40b4d04a3 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:47.051270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b524b-3a52-402e-84ae-e3a40b4d04a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:47.051689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b524b-3a52-402e-84ae-e3a40b4d04a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:47.053280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b524b-3a52-402e-84ae-e3a40b4d04a3 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:47.064376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 103/189] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:47.074409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b524b-3a52-402e-84ae-e3a40b4d04a3 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:47.134211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13f54-cf45-42a1-82bb-2e4502e551f3 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:47.177661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b76e9-9386-4a25-a045-75f2393664df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:47.178663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b76e9-9386-4a25-a045-75f2393664df None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:47.179180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b76e9-9386-4a25-a045-75f2393664df None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:47.179530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b76e9-9386-4a25-a045-75f2393664df None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:47.180290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b76e9-9386-4a25-a045-75f2393664df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:47.181821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b76e9-9386-4a25-a045-75f2393664df None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:47.205706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b524b-3a52-402e-84ae-e3a40b4d04a3 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:47.242848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf1791-a917-4169-bed7-f919b69b9b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:47.243505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf1791-a917-4169-bed7-f919b69b9b4a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:47.243769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf1791-a917-4169-bed7-f919b69b9b4a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:47.244043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf1791-a917-4169-bed7-f919b69b9b4a None admin] PATH_INFO: `/v3/users/a257395b62c049b19544830102161cd2` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:47.252921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:47.265781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf1791-a917-4169-bed7-f919b69b9b4a None admin] RBAC: Authorizing `identity:get_user(user_id=a257395b62c049b19544830102161cd2)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:47.265781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf1791-a917-4169-bed7-f919b69b9b4a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:47.265781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 104/190] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:47 2022] GET /identity/v3/users/a257395b62c049b19544830102161cd2 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:47.265781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf1791-a917-4169-bed7-f919b69b9b4a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:47.296241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 87/191] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:39:46 2022] POST /identity/v3/users => generated 297 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:39:47.296241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b76e9-9386-4a25-a045-75f2393664df None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:47.330970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:47.414758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf1791-a917-4169-bed7-f919b69b9b4a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:47.453982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b76e9-9386-4a25-a045-75f2393664df None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:47.468866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4503ec9-3380-4d23-94b7-65a5db6c02ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:47.470591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4503ec9-3380-4d23-94b7-65a5db6c02ed None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:47.471380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4503ec9-3380-4d23-94b7-65a5db6c02ed None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:47.471857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4503ec9-3380-4d23-94b7-65a5db6c02ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:47.491584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4503ec9-3380-4d23-94b7-65a5db6c02ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:47.497773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f473d-4fc4-49cf-8102-f5419e7c0258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:47.498724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f473d-4fc4-49cf-8102-f5419e7c0258 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:47.499187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f473d-4fc4-49cf-8102-f5419e7c0258 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:47.499642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f473d-4fc4-49cf-8102-f5419e7c0258 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-e4503ec9-3380-4d23-94b7-65a5db6c02ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:47.500082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:47.500994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:47.501787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 105/192] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:39:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:47.511158 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4503ec9-3380-4d23-94b7-65a5db6c02ed None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:47.597650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f473d-4fc4-49cf-8102-f5419e7c0258 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:47.600541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f473d-4fc4-49cf-8102-f5419e7c0258 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:47.626753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:47.631282 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:47.698038 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4503ec9-3380-4d23-94b7-65a5db6c02ed None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:47.743463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2ce89-c10d-4691-95f7-8f868aa5ec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:47.744087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2ce89-c10d-4691-95f7-8f868aa5ec65 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:47.744451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2ce89-c10d-4691-95f7-8f868aa5ec65 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:47.744730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2ce89-c10d-4691-95f7-8f868aa5ec65 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:47.744846 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 88/193] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:47.749051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2ce89-c10d-4691-95f7-8f868aa5ec65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:47.749368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2ce89-c10d-4691-95f7-8f868aa5ec65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:47.750836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2ce89-c10d-4691-95f7-8f868aa5ec65 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:47.756015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f473d-4fc4-49cf-8102-f5419e7c0258 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:47.763027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 106/194] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:39:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:47.769215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2ce89-c10d-4691-95f7-8f868aa5ec65 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:47.909936 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f473d-4fc4-49cf-8102-f5419e7c0258 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:47.916694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2ce89-c10d-4691-95f7-8f868aa5ec65 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:47.939852 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:47.946242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:47.961204 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13094d-232e-463c-b1e2-dd56f8de04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:47.961853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13094d-232e-463c-b1e2-dd56f8de04a9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:47.962134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13094d-232e-463c-b1e2-dd56f8de04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:47.962410 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13094d-232e-463c-b1e2-dd56f8de04a9 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:47.963328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e386f1-3398-46aa-8a72-a84887399682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:47.963618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13094d-232e-463c-b1e2-dd56f8de04a9 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:47.964427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e386f1-3398-46aa-8a72-a84887399682 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:47.964913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e386f1-3398-46aa-8a72-a84887399682 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:47.965193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13094d-232e-463c-b1e2-dd56f8de04a9 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:47.965387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e386f1-3398-46aa-8a72-a84887399682 None admin] PATH_INFO: `/v3/users/a257395b62c049b19544830102161cd2` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:47.967662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 89/195] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:47 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:47.968290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e386f1-3398-46aa-8a72-a84887399682 None admin] RBAC: Authorizing `identity:get_user(user_id=a257395b62c049b19544830102161cd2)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:47.970157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e386f1-3398-46aa-8a72-a84887399682 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:47.972934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 107/196] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:47 2022] GET /identity/v3/users/a257395b62c049b19544830102161cd2 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:47.973646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13094d-232e-463c-b1e2-dd56f8de04a9 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:47.978678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e386f1-3398-46aa-8a72-a84887399682 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:48.113856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e386f1-3398-46aa-8a72-a84887399682 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:48.132550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13094d-232e-463c-b1e2-dd56f8de04a9 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:48.185704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5397142-3822-4afc-89ae-758682303fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:48.186243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5397142-3822-4afc-89ae-758682303fb4 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:48.186520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5397142-3822-4afc-89ae-758682303fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:48.186789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5397142-3822-4afc-89ae-758682303fb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:48.203778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608406a-d684-4438-8b7c-92edff5f16cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:48.204419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608406a-d684-4438-8b7c-92edff5f16cc None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:48.204699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608406a-d684-4438-8b7c-92edff5f16cc None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:48.204970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608406a-d684-4438-8b7c-92edff5f16cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:48.205715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608406a-d684-4438-8b7c-92edff5f16cc None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=98f2008c4252437ea79b895753cd0c32, user.id=a257395b62c049b19544830102161cd2) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:48.206019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608406a-d684-4438-8b7c-92edff5f16cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:48.207473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608406a-d684-4438-8b7c-92edff5f16cc None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:48.220850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 90/197] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:47 2022] GET /identity/v3/role_assignments?user.id=a257395b62c049b19544830102161cd2&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:48.280299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:48.296568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5397142-3822-4afc-89ae-758682303fb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:48.298035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5397142-3822-4afc-89ae-758682303fb4 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:48.412687 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 108/198] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 11:39:48.418376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608406a-d684-4438-8b7c-92edff5f16cc None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:48.505127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:48.595758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608406a-d684-4438-8b7c-92edff5f16cc None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:48.640357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edd178-12f5-4633-9ee8-b7d02959f9cd None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:48.640978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edd178-12f5-4633-9ee8-b7d02959f9cd None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:48.641481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edd178-12f5-4633-9ee8-b7d02959f9cd None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:48.643348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 109/199] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:39:48.648416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3242b3-faf8-43b4-81f8-c7ff3d45ec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:48.649095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3242b3-faf8-43b4-81f8-c7ff3d45ec5d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:48.649415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3242b3-faf8-43b4-81f8-c7ff3d45ec5d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:48.649752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3242b3-faf8-43b4-81f8-c7ff3d45ec5d None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:48.650018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7379de9-ed6e-4700-9658-99b1177abe5a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:48.650620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7379de9-ed6e-4700-9658-99b1177abe5a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:48.651053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3242b3-faf8-43b4-81f8-c7ff3d45ec5d None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:48.651310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7379de9-ed6e-4700-9658-99b1177abe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:48.652626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3242b3-faf8-43b4-81f8-c7ff3d45ec5d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:48.656250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 91/200] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:48 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:48.664239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3242b3-faf8-43b4-81f8-c7ff3d45ec5d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:48.679263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b7379de9-ed6e-4700-9658-99b1177abe5a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:48.681296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b7379de9-ed6e-4700-9658-99b1177abe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:48.783077 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 110/201] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:39:48.873543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3242b3-faf8-43b4-81f8-c7ff3d45ec5d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:48.909258 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:48.924580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a169c4-8b5f-4649-81dc-4b2698f94b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:48.925885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a169c4-8b5f-4649-81dc-4b2698f94b3e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:48.926343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a169c4-8b5f-4649-81dc-4b2698f94b3e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:48.926801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a169c4-8b5f-4649-81dc-4b2698f94b3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:48.927683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a169c4-8b5f-4649-81dc-4b2698f94b3e None admin] RBAC: Adding query filter params (role.id=4bb04c67143448539c5af8d8bdd621f9, scope.project.id=98f2008c4252437ea79b895753cd0c32, user.id=a257395b62c049b19544830102161cd2) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:48.928142 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a169c4-8b5f-4649-81dc-4b2698f94b3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:48.929797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a169c4-8b5f-4649-81dc-4b2698f94b3e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:48.936672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 92/202] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:48 2022] GET /identity/v3/role_assignments?user.id=a257395b62c049b19544830102161cd2&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=4bb04c67143448539c5af8d8bdd621f9 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:49.145032 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7379de9-ed6e-4700-9658-99b1177abe5a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:49.172033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:49.270806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7379de9-ed6e-4700-9658-99b1177abe5a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:49.306935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a276b-904b-4448-b7a9-e2d1a353e740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:49.307578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a276b-904b-4448-b7a9-e2d1a353e740 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:49.307853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a276b-904b-4448-b7a9-e2d1a353e740 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:49.308124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a276b-904b-4448-b7a9-e2d1a353e740 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:49.319818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a276b-904b-4448-b7a9-e2d1a353e740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-571a276b-904b-4448-b7a9-e2d1a353e740 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:49.324633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:49.325668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:49.327052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 111/203] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:49.331808 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a169c4-8b5f-4649-81dc-4b2698f94b3e None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:49.461369 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a169c4-8b5f-4649-81dc-4b2698f94b3e None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:49.496123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2034cad-5db0-4366-9663-8024f682f9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:49.499126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2034cad-5db0-4366-9663-8024f682f9f7 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:49.499455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2034cad-5db0-4366-9663-8024f682f9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:49.499730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2034cad-5db0-4366-9663-8024f682f9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:49.500465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2034cad-5db0-4366-9663-8024f682f9f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:49.500769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2034cad-5db0-4366-9663-8024f682f9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:49.502168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2034cad-5db0-4366-9663-8024f682f9f7 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:49.511777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 93/204] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:49.518132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a276b-904b-4448-b7a9-e2d1a353e740 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:49.610111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:49.637445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a276b-904b-4448-b7a9-e2d1a353e740 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:49.674128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535e461-a5eb-4662-972a-71fed8d7b18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:49.674822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535e461-a5eb-4662-972a-71fed8d7b18f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:49.675099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535e461-a5eb-4662-972a-71fed8d7b18f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:49.675369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535e461-a5eb-4662-972a-71fed8d7b18f None admin] PATH_INFO: `/v3/users/d972b329d9a74fdaac32d9cf48c2e9dc` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:49.676539 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535e461-a5eb-4662-972a-71fed8d7b18f None admin] RBAC: Authorizing `identity:get_user(user_id=d972b329d9a74fdaac32d9cf48c2e9dc)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:49.678083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535e461-a5eb-4662-972a-71fed8d7b18f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:49.679726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 112/205] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:49 2022] GET /identity/v3/users/d972b329d9a74fdaac32d9cf48c2e9dc => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:49.686343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2034cad-5db0-4366-9663-8024f682f9f7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:49.745705 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:49.823964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2034cad-5db0-4366-9663-8024f682f9f7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:49.860546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90507319-0b8d-4718-80ef-e86b3c42f93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:49.861404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90507319-0b8d-4718-80ef-e86b3c42f93a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:49.861851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90507319-0b8d-4718-80ef-e86b3c42f93a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:49.862294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90507319-0b8d-4718-80ef-e86b3c42f93a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:49.949135 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90507319-0b8d-4718-80ef-e86b3c42f93a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:49.951142 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90507319-0b8d-4718-80ef-e86b3c42f93a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:50.020966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:50.084781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 94/206] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:50.091584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535e461-a5eb-4662-972a-71fed8d7b18f None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:50.195454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535e461-a5eb-4662-972a-71fed8d7b18f None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:50.234021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9df9b-8af9-4561-887f-ee7b899de2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:50.234697 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9df9b-8af9-4561-887f-ee7b899de2a9 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:50.234976 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9df9b-8af9-4561-887f-ee7b899de2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:50.235303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9df9b-8af9-4561-887f-ee7b899de2a9 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:50.236462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9df9b-8af9-4561-887f-ee7b899de2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:50.237908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9df9b-8af9-4561-887f-ee7b899de2a9 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:50.239826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 113/207] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:50 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:50.245870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90507319-0b8d-4718-80ef-e86b3c42f93a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:50.378408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90507319-0b8d-4718-80ef-e86b3c42f93a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:50.429684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba9cc7-0dd5-46a8-8ae8-0aa23e100548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:50.430367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba9cc7-0dd5-46a8-8ae8-0aa23e100548 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:50.430648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba9cc7-0dd5-46a8-8ae8-0aa23e100548 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:50.430922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba9cc7-0dd5-46a8-8ae8-0aa23e100548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:50.431707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba9cc7-0dd5-46a8-8ae8-0aa23e100548 None admin] RBAC: Adding query filter params (role.id=7ae225f5f02c4cefaeb687c582acfcca, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c, user.id=d972b329d9a74fdaac32d9cf48c2e9dc) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:50.432017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba9cc7-0dd5-46a8-8ae8-0aa23e100548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:50.433573 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba9cc7-0dd5-46a8-8ae8-0aa23e100548 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:50.445292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 95/208] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:50 2022] GET /identity/v3/role_assignments?user.id=d972b329d9a74fdaac32d9cf48c2e9dc&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=7ae225f5f02c4cefaeb687c582acfcca => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:50.467249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3a83-50f8-4076-b340-c05aeade0ab7 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:50.467581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3a83-50f8-4076-b340-c05aeade0ab7 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:50.467796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3a83-50f8-4076-b340-c05aeade0ab7 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:50.469414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 114/209] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:50.474940 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d2c5-787f-401c-870f-2d2c1d08af05 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:50.475411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d2c5-787f-401c-870f-2d2c1d08af05 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:50.475837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d2c5-787f-401c-870f-2d2c1d08af05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:50.506153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-fc90d2c5-787f-401c-870f-2d2c1d08af05 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:50.507641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-fc90d2c5-787f-401c-870f-2d2c1d08af05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:50.519531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:50.544201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e787f2a-f68f-4cec-ac73-92cf314f0b57 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:50.544838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e787f2a-f68f-4cec-ac73-92cf314f0b57 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:50.545072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e787f2a-f68f-4cec-ac73-92cf314f0b57 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:50.546454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 115/210] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:39:50.553399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fecc6-6799-4d27-ab3e-7f821cfcfa39 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:50.553755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fecc6-6799-4d27-ab3e-7f821cfcfa39 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:50.553950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fecc6-6799-4d27-ab3e-7f821cfcfa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:50.598947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c74fecc6-6799-4d27-ab3e-7f821cfcfa39 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:50.598947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c74fecc6-6799-4d27-ab3e-7f821cfcfa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:50.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 96/211] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:50.610523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:50.704791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 116/212] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:50.893356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90d2c5-787f-401c-870f-2d2c1d08af05 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:50.989155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e42d9-8304-4ead-b691-a59017b60907 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:50.989449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e42d9-8304-4ead-b691-a59017b60907 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:50.989676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e42d9-8304-4ead-b691-a59017b60907 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:50.991087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 117/213] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:39:50.998637 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecd6af-2fb3-402b-95df-100bb3b3a370 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:50.999141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecd6af-2fb3-402b-95df-100bb3b3a370 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:50.999654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecd6af-2fb3-402b-95df-100bb3b3a370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:51.014104 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90d2c5-787f-401c-870f-2d2c1d08af05 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:51.024947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c8ecd6af-2fb3-402b-95df-100bb3b3a370 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:51.026164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c8ecd6af-2fb3-402b-95df-100bb3b3a370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:51.057519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f397891-551f-4b0d-b8ee-a6f585385243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:51.058166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f397891-551f-4b0d-b8ee-a6f585385243 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:51.058449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f397891-551f-4b0d-b8ee-a6f585385243 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:51.058724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f397891-551f-4b0d-b8ee-a6f585385243 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:51.059912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f397891-551f-4b0d-b8ee-a6f585385243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:51.061435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f397891-551f-4b0d-b8ee-a6f585385243 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:51.063346 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 97/214] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:51.064218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f397891-551f-4b0d-b8ee-a6f585385243 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:51.077003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:51.108112 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 118/215] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:51.113086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ecd6af-2fb3-402b-95df-100bb3b3a370 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:51.185098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f397891-551f-4b0d-b8ee-a6f585385243 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:51.197056 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:51.222708 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe35701-f361-4ba4-b8e5-d9ece7ffe0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:51.223337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe35701-f361-4ba4-b8e5-d9ece7ffe0c5 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:51.223612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe35701-f361-4ba4-b8e5-d9ece7ffe0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:51.223932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe35701-f361-4ba4-b8e5-d9ece7ffe0c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:51.236566 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe35701-f361-4ba4-b8e5-d9ece7ffe0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:51.237719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ecd6af-2fb3-402b-95df-100bb3b3a370 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-fbe35701-f361-4ba4-b8e5-d9ece7ffe0c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:51.242421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:51.243317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:51.244127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 98/216] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:51.251409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe35701-f361-4ba4-b8e5-d9ece7ffe0c5 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:51.278628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da832f-d9b6-41a1-9956-0efbeed0caae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:51.279294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da832f-d9b6-41a1-9956-0efbeed0caae None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:51.279572 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da832f-d9b6-41a1-9956-0efbeed0caae None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:51.279837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da832f-d9b6-41a1-9956-0efbeed0caae None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:51.280547 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da832f-d9b6-41a1-9956-0efbeed0caae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:51.282175 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da832f-d9b6-41a1-9956-0efbeed0caae None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:51.338012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:51.370000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe35701-f361-4ba4-b8e5-d9ece7ffe0c5 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:51.398743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 119/217] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:39:51 2022] POST /identity/v3/users => generated 297 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:39:51.400516 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da832f-d9b6-41a1-9956-0efbeed0caae None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:51.411500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4788ba3-92d8-4f23-8200-323b6d8b64bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:51.412212 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4788ba3-92d8-4f23-8200-323b6d8b64bf None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:51.412503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4788ba3-92d8-4f23-8200-323b6d8b64bf None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:51.412779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4788ba3-92d8-4f23-8200-323b6d8b64bf None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:51.413604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4788ba3-92d8-4f23-8200-323b6d8b64bf None admin] RBAC: Adding query filter params (name=member) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:51.413913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4788ba3-92d8-4f23-8200-323b6d8b64bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:51.415347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4788ba3-92d8-4f23-8200-323b6d8b64bf None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:51.430809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 99/218] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:51.432250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4788ba3-92d8-4f23-8200-323b6d8b64bf None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:51.551592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da832f-d9b6-41a1-9956-0efbeed0caae None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:51.568533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4788ba3-92d8-4f23-8200-323b6d8b64bf None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:51.591097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:51.601327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7a8b1-79fc-481a-854b-0a278e921996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:51.603921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:51.604032 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7a8b1-79fc-481a-854b-0a278e921996 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:51.604032 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7a8b1-79fc-481a-854b-0a278e921996 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:51.604032 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7a8b1-79fc-481a-854b-0a278e921996 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:51.609112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad26158-6ff6-4ad5-870e-b059cf28976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:51.609754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad26158-6ff6-4ad5-870e-b059cf28976f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:51.610034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad26158-6ff6-4ad5-870e-b059cf28976f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:51.610312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad26158-6ff6-4ad5-870e-b059cf28976f None admin] PATH_INFO: `/v3/users/c4a3748f073540d68452a90d491aea72` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:51.616711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7a8b1-79fc-481a-854b-0a278e921996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-9ae7a8b1-79fc-481a-854b-0a278e921996 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:51.622796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:51.623720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:51.625004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 120/219] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:39:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:51.630307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad26158-6ff6-4ad5-870e-b059cf28976f None admin] RBAC: Authorizing `identity:get_user(user_id=c4a3748f073540d68452a90d491aea72)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:51.630663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7a8b1-79fc-481a-854b-0a278e921996 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:51.631989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad26158-6ff6-4ad5-870e-b059cf28976f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:51.633823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 100/220] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:51 2022] GET /identity/v3/users/c4a3748f073540d68452a90d491aea72 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:51.639531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad26158-6ff6-4ad5-870e-b059cf28976f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:51.765857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7a8b1-79fc-481a-854b-0a278e921996 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:51.768581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad26158-6ff6-4ad5-870e-b059cf28976f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:51.814210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace768a-ea17-4de0-a8d4-071658757faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:51.814730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace768a-ea17-4de0-a8d4-071658757faa None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:51.815029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace768a-ea17-4de0-a8d4-071658757faa None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:51.815277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace768a-ea17-4de0-a8d4-071658757faa None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:51.815951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace768a-ea17-4de0-a8d4-071658757faa None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:51.816509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace768a-ea17-4de0-a8d4-071658757faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:51.817734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace768a-ea17-4de0-a8d4-071658757faa None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:51.826037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c17602-06ea-48c4-a858-c8e41682068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:51.826883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c17602-06ea-48c4-a858-c8e41682068d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:51.827311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c17602-06ea-48c4-a858-c8e41682068d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:51.827660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 121/221] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:51.828015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c17602-06ea-48c4-a858-c8e41682068d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:51.835123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace768a-ea17-4de0-a8d4-071658757faa None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:51.899936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:51.911053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:51.924556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c17602-06ea-48c4-a858-c8e41682068d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:51.927196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c17602-06ea-48c4-a858-c8e41682068d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:51.972401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace768a-ea17-4de0-a8d4-071658757faa None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:52.019223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e97696-ed80-4d6c-936d-ec9c759412fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:52.019773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e97696-ed80-4d6c-936d-ec9c759412fd None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:52.020050 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e97696-ed80-4d6c-936d-ec9c759412fd None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:52.020350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e97696-ed80-4d6c-936d-ec9c759412fd None admin] PATH_INFO: `/v3/users/a257395b62c049b19544830102161cd2` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:52.021778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e97696-ed80-4d6c-936d-ec9c759412fd None admin] RBAC: Authorizing `identity:get_user(user_id=a257395b62c049b19544830102161cd2)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:52.023259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e97696-ed80-4d6c-936d-ec9c759412fd None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:52.025195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 122/222] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:51 2022] GET /identity/v3/users/a257395b62c049b19544830102161cd2 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:52.030978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e97696-ed80-4d6c-936d-ec9c759412fd None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:52.074638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 101/223] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:52.082371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe974112-6151-4814-88db-231345f6b586 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:52.082753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe974112-6151-4814-88db-231345f6b586 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:52.083104 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe974112-6151-4814-88db-231345f6b586 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:52.084697 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 102/224] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:39:52.085327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe974112-6151-4814-88db-231345f6b586 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:52.164246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e97696-ed80-4d6c-936d-ec9c759412fd None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:52.187170 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:52.197760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe974112-6151-4814-88db-231345f6b586 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:52.234816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e03da6-94e3-4dfd-8719-2de01dae870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:52.235571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e03da6-94e3-4dfd-8719-2de01dae870c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:52.235919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e03da6-94e3-4dfd-8719-2de01dae870c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:52.236296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e03da6-94e3-4dfd-8719-2de01dae870c None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:52.237621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e03da6-94e3-4dfd-8719-2de01dae870c None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:52.239220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e03da6-94e3-4dfd-8719-2de01dae870c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:52.241004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 103/225] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:52 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:52.242173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:52.242286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27cdcf-edcb-4962-8f9c-e72054e440d3 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:52.242626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27cdcf-edcb-4962-8f9c-e72054e440d3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:52.243214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27cdcf-edcb-4962-8f9c-e72054e440d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:52.261982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7b27cdcf-edcb-4962-8f9c-e72054e440d3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:52.263827 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7b27cdcf-edcb-4962-8f9c-e72054e440d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:52.265324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0199e-78ed-47c9-8991-21683555291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:52.266455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0199e-78ed-47c9-8991-21683555291b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:52.266918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0199e-78ed-47c9-8991-21683555291b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:52.268328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0199e-78ed-47c9-8991-21683555291b None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:52.269728 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0199e-78ed-47c9-8991-21683555291b None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:52.271381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0199e-78ed-47c9-8991-21683555291b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:52.273824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 123/226] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:52 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:52.274971 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0199e-78ed-47c9-8991-21683555291b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:52.345856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 104/227] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:52.347155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27cdcf-edcb-4962-8f9c-e72054e440d3 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:52.432550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0199e-78ed-47c9-8991-21683555291b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:52.486804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:52.498743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77764e8-752c-44f9-b0e9-fd57be8ca4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:52.499359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77764e8-752c-44f9-b0e9-fd57be8ca4aa None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:52.499635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77764e8-752c-44f9-b0e9-fd57be8ca4aa None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:52.501683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77764e8-752c-44f9-b0e9-fd57be8ca4aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:52.501683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77764e8-752c-44f9-b0e9-fd57be8ca4aa None admin] RBAC: Adding query filter params (role.id=3f0c34a1f93e49adb1757f315333cc48, scope.project.id=98f2008c4252437ea79b895753cd0c32, user.id=c4a3748f073540d68452a90d491aea72) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:52.501683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77764e8-752c-44f9-b0e9-fd57be8ca4aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:52.502757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77764e8-752c-44f9-b0e9-fd57be8ca4aa None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:52.510895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27cdcf-edcb-4962-8f9c-e72054e440d3 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:52.515116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 124/228] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:52 2022] GET /identity/v3/role_assignments?user.id=c4a3748f073540d68452a90d491aea72&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:52.575115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44570e-5fec-41fe-8861-9ab1683043c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:52.575815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44570e-5fec-41fe-8861-9ab1683043c9 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:52.576097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44570e-5fec-41fe-8861-9ab1683043c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:52.576398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44570e-5fec-41fe-8861-9ab1683043c9 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32/users/a257395b62c049b19544830102161cd2/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:52.578616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44570e-5fec-41fe-8861-9ab1683043c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=98f2008c4252437ea79b895753cd0c32, user_id=a257395b62c049b19544830102161cd2, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:52.580127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44570e-5fec-41fe-8861-9ab1683043c9 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:52.661336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 105/229] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:39:52 2022] PUT /identity/v3/projects/98f2008c4252437ea79b895753cd0c32/users/a257395b62c049b19544830102161cd2/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:39:52.696417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77764e8-752c-44f9-b0e9-fd57be8ca4aa None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:52.792785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:52.869739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77764e8-752c-44f9-b0e9-fd57be8ca4aa None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:52.915485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d66b2-88dd-4678-886e-c89055276cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:52.916069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d66b2-88dd-4678-886e-c89055276cdb None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:52.916373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d66b2-88dd-4678-886e-c89055276cdb None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:52.916652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d66b2-88dd-4678-886e-c89055276cdb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:52.935441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d66b2-88dd-4678-886e-c89055276cdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-5a1d66b2-88dd-4678-886e-c89055276cdb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:52.941034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:52.941955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:52.943153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 125/230] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:39:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:52.950588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44570e-5fec-41fe-8861-9ab1683043c9 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:53.057713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44570e-5fec-41fe-8861-9ab1683043c9 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:53.100554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db07a4f-3e0c-478f-b6ae-ffe459e07217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:53.101189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db07a4f-3e0c-478f-b6ae-ffe459e07217 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:53.101462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db07a4f-3e0c-478f-b6ae-ffe459e07217 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:53.101739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db07a4f-3e0c-478f-b6ae-ffe459e07217 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:53.102442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db07a4f-3e0c-478f-b6ae-ffe459e07217 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:53.102750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db07a4f-3e0c-478f-b6ae-ffe459e07217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:53.104225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db07a4f-3e0c-478f-b6ae-ffe459e07217 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:53.114879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 106/231] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:39:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:53.121101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d66b2-88dd-4678-886e-c89055276cdb None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:53.247308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d66b2-88dd-4678-886e-c89055276cdb None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:53.263620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:53.291590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478ec79-82c8-4f57-8861-fa1be780b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:53.292291 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478ec79-82c8-4f57-8861-fa1be780b930 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:53.292577 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478ec79-82c8-4f57-8861-fa1be780b930 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:53.292851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478ec79-82c8-4f57-8861-fa1be780b930 None admin] PATH_INFO: `/v3/users/a257395b62c049b19544830102161cd2` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:53.294071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478ec79-82c8-4f57-8861-fa1be780b930 None admin] RBAC: Authorizing `identity:get_user(user_id=a257395b62c049b19544830102161cd2)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:53.295534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478ec79-82c8-4f57-8861-fa1be780b930 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:53.297147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 126/232] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:53 2022] GET /identity/v3/users/a257395b62c049b19544830102161cd2 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:53.302998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db07a4f-3e0c-478f-b6ae-ffe459e07217 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:53.323845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:53.447528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db07a4f-3e0c-478f-b6ae-ffe459e07217 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:53.489792 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585428a-47f3-4188-912c-e79cd95e86a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:53.490426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585428a-47f3-4188-912c-e79cd95e86a1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:53.490708 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585428a-47f3-4188-912c-e79cd95e86a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:53.490986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585428a-47f3-4188-912c-e79cd95e86a1 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:53.492224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585428a-47f3-4188-912c-e79cd95e86a1 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:53.493819 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585428a-47f3-4188-912c-e79cd95e86a1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:53.495774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 107/233] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:53 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:53.501720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478ec79-82c8-4f57-8861-fa1be780b930 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:53.653522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478ec79-82c8-4f57-8861-fa1be780b930 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:53.689601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e60855-6fa0-4476-b17f-181f232e1388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:53.690634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60855-6fa0-4476-b17f-181f232e1388 None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:53.691126 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60855-6fa0-4476-b17f-181f232e1388 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:53.691627 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60855-6fa0-4476-b17f-181f232e1388 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32/users/a257395b62c049b19544830102161cd2/roles/4bb04c67143448539c5af8d8bdd621f9` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:53.694085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e60855-6fa0-4476-b17f-181f232e1388 None admin] RBAC: Authorizing `identity:create_grant(project_id=98f2008c4252437ea79b895753cd0c32, user_id=a257395b62c049b19544830102161cd2, role_id=4bb04c67143448539c5af8d8bdd621f9)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:53.695785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e60855-6fa0-4476-b17f-181f232e1388 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:53.803905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 127/234] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:39:53 2022] PUT /identity/v3/projects/98f2008c4252437ea79b895753cd0c32/users/a257395b62c049b19544830102161cd2/roles/4bb04c67143448539c5af8d8bdd621f9 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:39:54.311477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e626f1f-9712-444f-85d8-c5ff768c3596 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:54.312002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e626f1f-9712-444f-85d8-c5ff768c3596 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:54.312081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e626f1f-9712-444f-85d8-c5ff768c3596 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:54.313704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 108/235] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:54.318939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0489f-5559-423c-b486-49c47fc8fe03 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:54.319423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0489f-5559-423c-b486-49c47fc8fe03 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:54.319502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0489f-5559-423c-b486-49c47fc8fe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:54.340926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d9e0489f-5559-423c-b486-49c47fc8fe03 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:54.342191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d9e0489f-5559-423c-b486-49c47fc8fe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:54.354522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bce32-629b-4cb6-9ab5-03c3f7e1d984 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:54.354874 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bce32-629b-4cb6-9ab5-03c3f7e1d984 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:54.355176 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bce32-629b-4cb6-9ab5-03c3f7e1d984 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:54.357594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 109/236] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:54.368538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce0086-3002-4df2-bf17-71c0acd64683 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:54.369165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce0086-3002-4df2-bf17-71c0acd64683 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:54.370089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce0086-3002-4df2-bf17-71c0acd64683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:54.380807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:54.405162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-00ce0086-3002-4df2-bf17-71c0acd64683 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:54.407753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-00ce0086-3002-4df2-bf17-71c0acd64683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:54.445643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 128/237] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:54.521643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 110/238] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:39:54.793625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0489f-5559-423c-b486-49c47fc8fe03 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:54.921457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce0086-3002-4df2-bf17-71c0acd64683 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:54.925947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0489f-5559-423c-b486-49c47fc8fe03 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:54.932248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:54.992699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beade15b-44e1-49a4-a854-e53eed7a5a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:54.993932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beade15b-44e1-49a4-a854-e53eed7a5a2e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:54.995398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beade15b-44e1-49a4-a854-e53eed7a5a2e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.000632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beade15b-44e1-49a4-a854-e53eed7a5a2e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.020763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beade15b-44e1-49a4-a854-e53eed7a5a2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-beade15b-44e1-49a4-a854-e53eed7a5a2e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:55.027393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:55.028377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:39:55.029188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:55.035249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 129/239] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:55.042228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beade15b-44e1-49a4-a854-e53eed7a5a2e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:55.076669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:55.102004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce0086-3002-4df2-bf17-71c0acd64683 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:55.155721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beade15b-44e1-49a4-a854-e53eed7a5a2e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:55.194614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df46d1-e576-45e4-9537-dc8ca386d0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:55.195517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df46d1-e576-45e4-9537-dc8ca386d0d4 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:55.195864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df46d1-e576-45e4-9537-dc8ca386d0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.196267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df46d1-e576-45e4-9537-dc8ca386d0d4 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.197137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df46d1-e576-45e4-9537-dc8ca386d0d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:55.197541 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df46d1-e576-45e4-9537-dc8ca386d0d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:55.199027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df46d1-e576-45e4-9537-dc8ca386d0d4 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:55.208161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 130/240] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:55.212933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4b207-fb15-4fc7-a6d1-1b8f74f4cb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:55.213612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4b207-fb15-4fc7-a6d1-1b8f74f4cb0b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:55.213873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df46d1-e576-45e4-9537-dc8ca386d0d4 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:55.213975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4b207-fb15-4fc7-a6d1-1b8f74f4cb0b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.214173 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4b207-fb15-4fc7-a6d1-1b8f74f4cb0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.215454 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4b207-fb15-4fc7-a6d1-1b8f74f4cb0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:55.217108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4b207-fb15-4fc7-a6d1-1b8f74f4cb0b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:55.218999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 111/241] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:55.235014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4b207-fb15-4fc7-a6d1-1b8f74f4cb0b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:55.320620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:55.332956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df46d1-e576-45e4-9537-dc8ca386d0d4 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:55.379537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4bc87-85d1-4a67-a733-5578788fd4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:55.380205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4bc87-85d1-4a67-a733-5578788fd4ba None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:55.380501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4bc87-85d1-4a67-a733-5578788fd4ba None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.380773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4bc87-85d1-4a67-a733-5578788fd4ba None admin] PATH_INFO: `/v3/users/acfd310ad89b4fcaa36546c7df9c8319` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.393569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:55.397415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4bc87-85d1-4a67-a733-5578788fd4ba None admin] RBAC: Authorizing `identity:get_user(user_id=acfd310ad89b4fcaa36546c7df9c8319)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:55.399177 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4bc87-85d1-4a67-a733-5578788fd4ba None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:55.401220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 131/242] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:55 2022] GET /identity/v3/users/acfd310ad89b4fcaa36546c7df9c8319 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:55.402500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba764cf6-8afc-4758-91e7-331243f218c8 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:55.402838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba764cf6-8afc-4758-91e7-331243f218c8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.403117 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba764cf6-8afc-4758-91e7-331243f218c8 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.418729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 132/243] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:55.418729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba764cf6-8afc-4758-91e7-331243f218c8 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:55.426053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4b207-fb15-4fc7-a6d1-1b8f74f4cb0b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:55.495556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d325c-8654-47bd-bee3-b0160993c145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:55.497501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d325c-8654-47bd-bee3-b0160993c145 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:55.498243 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d325c-8654-47bd-bee3-b0160993c145 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.498908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d325c-8654-47bd-bee3-b0160993c145 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.500320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d325c-8654-47bd-bee3-b0160993c145 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:55.503183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d325c-8654-47bd-bee3-b0160993c145 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:55.534099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba764cf6-8afc-4758-91e7-331243f218c8 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:55.581202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebfce1-e679-40e1-a99b-b5581adf0175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:55.581771 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebfce1-e679-40e1-a99b-b5581adf0175 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:55.582064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebfce1-e679-40e1-a99b-b5581adf0175 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.582319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebfce1-e679-40e1-a99b-b5581adf0175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.602259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 112/244] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:39:55 2022] POST /identity/v3/users => generated 293 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:39:55.616698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:55.616840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea04800-d23a-41ab-8aa6-950dd69727e2 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:55.616840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea04800-d23a-41ab-8aa6-950dd69727e2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.616840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea04800-d23a-41ab-8aa6-950dd69727e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.657462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-aea04800-d23a-41ab-8aa6-950dd69727e2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:55.658813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-aea04800-d23a-41ab-8aa6-950dd69727e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:55.666091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebfce1-e679-40e1-a99b-b5581adf0175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:55.667062 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebfce1-e679-40e1-a99b-b5581adf0175 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:55.767596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 113/245] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:55.769445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cfd5d-e8fc-4152-84de-9a1871d2b6c8 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:55.769805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cfd5d-e8fc-4152-84de-9a1871d2b6c8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.770131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cfd5d-e8fc-4152-84de-9a1871d2b6c8 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.776280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 114/246] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:55.781877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9dcf1-3de6-465c-8ea4-a6f18c308077 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:55.782280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9dcf1-3de6-465c-8ea4-a6f18c308077 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.782615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9dcf1-3de6-465c-8ea4-a6f18c308077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.811922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 133/247] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:55.820988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-cea9dcf1-3de6-465c-8ea4-a6f18c308077 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:55.822342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cea9dcf1-3de6-465c-8ea4-a6f18c308077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:55.827621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebfce1-e679-40e1-a99b-b5581adf0175 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:55.887321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:55.903777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:55.916899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 115/248] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:55.947419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebfce1-e679-40e1-a99b-b5581adf0175 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:55.987933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b0e48-f711-497e-a1cf-bdc3c49260e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:55.988686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b0e48-f711-497e-a1cf-bdc3c49260e1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:55.988966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b0e48-f711-497e-a1cf-bdc3c49260e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:55.989233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b0e48-f711-497e-a1cf-bdc3c49260e1 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:55.990396 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b0e48-f711-497e-a1cf-bdc3c49260e1 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:55.991852 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b0e48-f711-497e-a1cf-bdc3c49260e1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:55.993540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 134/249] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:55 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:55.999523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9dcf1-3de6-465c-8ea4-a6f18c308077 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:56.068298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b0e48-f711-497e-a1cf-bdc3c49260e1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:56.185570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9dcf1-3de6-465c-8ea4-a6f18c308077 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:56.251749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b0e48-f711-497e-a1cf-bdc3c49260e1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:56.287441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:56.313229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2848f-7a69-4e79-ab94-cbb5bfb3c4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:56.313872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2848f-7a69-4e79-ab94-cbb5bfb3c4c0 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:56.314148 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2848f-7a69-4e79-ab94-cbb5bfb3c4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:56.314418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2848f-7a69-4e79-ab94-cbb5bfb3c4c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:56.316524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4570d99-7259-4cdd-a6e6-1369e08c4236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:56.317175 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4570d99-7259-4cdd-a6e6-1369e08c4236 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:56.317424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4570d99-7259-4cdd-a6e6-1369e08c4236 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:56.317699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4570d99-7259-4cdd-a6e6-1369e08c4236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:56.318446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4570d99-7259-4cdd-a6e6-1369e08c4236 None admin] RBAC: Adding query filter params (role.id=7ae225f5f02c4cefaeb687c582acfcca, scope.project.id=98f2008c4252437ea79b895753cd0c32, user.id=acfd310ad89b4fcaa36546c7df9c8319) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:56.318755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4570d99-7259-4cdd-a6e6-1369e08c4236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:56.320273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4570d99-7259-4cdd-a6e6-1369e08c4236 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:56.326466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 116/250] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:55 2022] GET /identity/v3/role_assignments?user.id=acfd310ad89b4fcaa36546c7df9c8319&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=7ae225f5f02c4cefaeb687c582acfcca => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:56.329459 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2848f-7a69-4e79-ab94-cbb5bfb3c4c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:56.333181 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4570d99-7259-4cdd-a6e6-1369e08c4236 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-7ed2848f-7a69-4e79-ab94-cbb5bfb3c4c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:56.335534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:56.336495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:56.337292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:39:56.339649 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 135/251] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:56.349218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2848f-7a69-4e79-ab94-cbb5bfb3c4c0 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:56.420628 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:56.505734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4570d99-7259-4cdd-a6e6-1369e08c4236 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:56.513028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2848f-7a69-4e79-ab94-cbb5bfb3c4c0 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:56.544432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c7a08-58b6-4770-9d22-aac0d1306d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:56.545653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c7a08-58b6-4770-9d22-aac0d1306d50 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:56.546099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c7a08-58b6-4770-9d22-aac0d1306d50 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:56.546509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c7a08-58b6-4770-9d22-aac0d1306d50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:56.560361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c7a08-58b6-4770-9d22-aac0d1306d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-452c7a08-58b6-4770-9d22-aac0d1306d50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:56.566806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:56.567960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:39:56.568870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:56.570487 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 117/252] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:39:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:56.576978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c7a08-58b6-4770-9d22-aac0d1306d50 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:56.624941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b712fbc-2b05-49d6-8370-d622c77eff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:56.625580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b712fbc-2b05-49d6-8370-d622c77eff4c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:56.625866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b712fbc-2b05-49d6-8370-d622c77eff4c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:56.626145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b712fbc-2b05-49d6-8370-d622c77eff4c None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:56.626841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b712fbc-2b05-49d6-8370-d622c77eff4c None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:56.627150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b712fbc-2b05-49d6-8370-d622c77eff4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:56.628641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b712fbc-2b05-49d6-8370-d622c77eff4c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:56.650199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 136/253] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:56.656536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b712fbc-2b05-49d6-8370-d622c77eff4c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:56.706810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c7a08-58b6-4770-9d22-aac0d1306d50 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:56.725765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:56.748565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:56.751546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23292450-fe35-4814-a71a-5d100c7467da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:56.752426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23292450-fe35-4814-a71a-5d100c7467da None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:56.753706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23292450-fe35-4814-a71a-5d100c7467da None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:56.754189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23292450-fe35-4814-a71a-5d100c7467da None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:56.755082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23292450-fe35-4814-a71a-5d100c7467da None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:56.756571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23292450-fe35-4814-a71a-5d100c7467da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:56.758334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23292450-fe35-4814-a71a-5d100c7467da None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:56.781144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 118/254] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:56.787687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a8f9e-90b5-4471-85ed-c07ed89db87b None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:56.788006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a8f9e-90b5-4471-85ed-c07ed89db87b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:56.800375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a8f9e-90b5-4471-85ed-c07ed89db87b None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:56.801932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 119/255] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:56 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:56.802745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a8f9e-90b5-4471-85ed-c07ed89db87b None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:56.861264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b712fbc-2b05-49d6-8370-d622c77eff4c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:56.947731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafc44d-d744-40df-94e2-bcebae6e1ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:56.948403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafc44d-d744-40df-94e2-bcebae6e1ef9 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:56.948700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafc44d-d744-40df-94e2-bcebae6e1ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:56.948993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafc44d-d744-40df-94e2-bcebae6e1ef9 None admin] PATH_INFO: `/v3/users/c4a3748f073540d68452a90d491aea72` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:56.953027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafc44d-d744-40df-94e2-bcebae6e1ef9 None admin] RBAC: Authorizing `identity:get_user(user_id=c4a3748f073540d68452a90d491aea72)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:56.954561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafc44d-d744-40df-94e2-bcebae6e1ef9 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:56.956381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 137/256] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:56 2022] GET /identity/v3/users/c4a3748f073540d68452a90d491aea72 => generated 297 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:56.957389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f175592-216f-4f63-bcf4-f6ff4aaad0b7 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:56.957708 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f175592-216f-4f63-bcf4-f6ff4aaad0b7 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:56.957947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f175592-216f-4f63-bcf4-f6ff4aaad0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:56.970449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a8f9e-90b5-4471-85ed-c07ed89db87b None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:56.993989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2f175592-216f-4f63-bcf4-f6ff4aaad0b7 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:56.995181 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2f175592-216f-4f63-bcf4-f6ff4aaad0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:57.025037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1cc33-e394-4606-a280-171e02e67ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:57.025614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1cc33-e394-4606-a280-171e02e67ca2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:57.025898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1cc33-e394-4606-a280-171e02e67ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:57.026178 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1cc33-e394-4606-a280-171e02e67ca2 None admin] PATH_INFO: `/v3/users/a257395b62c049b19544830102161cd2` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:57.027421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1cc33-e394-4606-a280-171e02e67ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=a257395b62c049b19544830102161cd2)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:57.029000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1cc33-e394-4606-a280-171e02e67ca2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:57.030629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 120/257] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:56 2022] GET /identity/v3/users/a257395b62c049b19544830102161cd2 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:57.031702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1cc33-e394-4606-a280-171e02e67ca2 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:57.116914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 138/258] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:57.118196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f175592-216f-4f63-bcf4-f6ff4aaad0b7 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:57.147440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:57.178370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:57.213460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1cc33-e394-4606-a280-171e02e67ca2 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:57.284210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a3f87-72b0-4fcd-af46-0b1ce24502c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:57.285182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a3f87-72b0-4fcd-af46-0b1ce24502c4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:57.285659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a3f87-72b0-4fcd-af46-0b1ce24502c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:57.286144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a3f87-72b0-4fcd-af46-0b1ce24502c4 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:57.293069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a3f87-72b0-4fcd-af46-0b1ce24502c4 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:57.295358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a3f87-72b0-4fcd-af46-0b1ce24502c4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:57.297651 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 121/259] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:57 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:57.303068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a3f87-72b0-4fcd-af46-0b1ce24502c4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:57.368045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f175592-216f-4f63-bcf4-f6ff4aaad0b7 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:57.441600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dee57-f639-4487-94b9-c21c13d338cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:57.442229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dee57-f639-4487-94b9-c21c13d338cd None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:57.442551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dee57-f639-4487-94b9-c21c13d338cd None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:57.442827 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dee57-f639-4487-94b9-c21c13d338cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:57.464374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a3f87-72b0-4fcd-af46-0b1ce24502c4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:57.481727 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:57.532493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fafed7-5d6f-43e1-984f-5fa12ab534a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:57.533707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fafed7-5d6f-43e1-984f-5fa12ab534a7 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:57.534195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fafed7-5d6f-43e1-984f-5fa12ab534a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:57.534778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fafed7-5d6f-43e1-984f-5fa12ab534a7 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32/users/c4a3748f073540d68452a90d491aea72/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:57.537053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fafed7-5d6f-43e1-984f-5fa12ab534a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=98f2008c4252437ea79b895753cd0c32, user_id=c4a3748f073540d68452a90d491aea72, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:57.538980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fafed7-5d6f-43e1-984f-5fa12ab534a7 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:57.561358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421dee57-f639-4487-94b9-c21c13d338cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:57.563190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421dee57-f639-4487-94b9-c21c13d338cd None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:57.609354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 122/260] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:39:57 2022] PUT /identity/v3/projects/98f2008c4252437ea79b895753cd0c32/users/c4a3748f073540d68452a90d491aea72/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:39:57.610778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fafed7-5d6f-43e1-984f-5fa12ab534a7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:57.617343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:57.716012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 139/261] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:57.729196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dee57-f639-4487-94b9-c21c13d338cd None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:57.812042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fafed7-5d6f-43e1-984f-5fa12ab534a7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:57.862720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bbba1-8fb1-4bef-ae11-780517e1233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:57.863741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bbba1-8fb1-4bef-ae11-780517e1233c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:57.864250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bbba1-8fb1-4bef-ae11-780517e1233c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:57.864728 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bbba1-8fb1-4bef-ae11-780517e1233c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:57.886864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dee57-f639-4487-94b9-c21c13d338cd None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:57.893252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bbba1-8fb1-4bef-ae11-780517e1233c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-4b0bbba1-8fb1-4bef-ae11-780517e1233c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:57.898693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:57.899564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:39:57.900403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:57.912200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 123/262] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:39:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:39:57.914318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bbba1-8fb1-4bef-ae11-780517e1233c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:57.941524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b44e73-2110-4339-bb80-8a65001113f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:57.942161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b44e73-2110-4339-bb80-8a65001113f2 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:57.942436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b44e73-2110-4339-bb80-8a65001113f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:57.942701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b44e73-2110-4339-bb80-8a65001113f2 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:57.948162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b44e73-2110-4339-bb80-8a65001113f2 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:57.949724 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b44e73-2110-4339-bb80-8a65001113f2 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:57.952363 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 140/263] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:57 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:57.974894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b44e73-2110-4339-bb80-8a65001113f2 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:57.978488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:57.982687 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:58.066486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bbba1-8fb1-4bef-ae11-780517e1233c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:58.110873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bba1bb-1aa3-4087-b4a2-da00bbe3d2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:58.111756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bba1bb-1aa3-4087-b4a2-da00bbe3d2ca None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:58.113361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bba1bb-1aa3-4087-b4a2-da00bbe3d2ca None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:58.113809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bba1bb-1aa3-4087-b4a2-da00bbe3d2ca None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:58.114967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bba1bb-1aa3-4087-b4a2-da00bbe3d2ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:58.119139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bba1bb-1aa3-4087-b4a2-da00bbe3d2ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:58.121016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bba1bb-1aa3-4087-b4a2-da00bbe3d2ca None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:58.154591 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 124/264] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:39:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:58.164251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bba1bb-1aa3-4087-b4a2-da00bbe3d2ca None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:58.239417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b44e73-2110-4339-bb80-8a65001113f2 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:58.336968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bba1bb-1aa3-4087-b4a2-da00bbe3d2ca None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:58.343127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:58.354308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ce38c-1365-488d-986b-3958c8277a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:58.355356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ce38c-1365-488d-986b-3958c8277a15 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:58.355978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ce38c-1365-488d-986b-3958c8277a15 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:58.356499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ce38c-1365-488d-986b-3958c8277a15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:58.363522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ce38c-1365-488d-986b-3958c8277a15 None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=98f2008c4252437ea79b895753cd0c32, user.id=a257395b62c049b19544830102161cd2) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:58.363522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ce38c-1365-488d-986b-3958c8277a15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:58.363522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ce38c-1365-488d-986b-3958c8277a15 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:58.379344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 141/265] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:57 2022] GET /identity/v3/role_assignments?user.id=a257395b62c049b19544830102161cd2&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 608 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:58.411357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830e094-9f05-4e49-9c57-6c0a67489203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:58.412395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830e094-9f05-4e49-9c57-6c0a67489203 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:58.412934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830e094-9f05-4e49-9c57-6c0a67489203 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:58.413426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830e094-9f05-4e49-9c57-6c0a67489203 None admin] PATH_INFO: `/v3/users/a257395b62c049b19544830102161cd2` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:58.414913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830e094-9f05-4e49-9c57-6c0a67489203 None admin] RBAC: Authorizing `identity:get_user(user_id=a257395b62c049b19544830102161cd2)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:58.416698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830e094-9f05-4e49-9c57-6c0a67489203 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:58.418890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 125/266] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:39:58 2022] GET /identity/v3/users/a257395b62c049b19544830102161cd2 => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:58.425156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ce38c-1365-488d-986b-3958c8277a15 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:58.524942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:58.569480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ce38c-1365-488d-986b-3958c8277a15 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:58.610947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193275b-1c2c-4112-893c-863307c4df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:58.611957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193275b-1c2c-4112-893c-863307c4df7d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:58.612500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193275b-1c2c-4112-893c-863307c4df7d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:58.613026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193275b-1c2c-4112-893c-863307c4df7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:58.712571 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193275b-1c2c-4112-893c-863307c4df7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:58.714066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193275b-1c2c-4112-893c-863307c4df7d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:58.805376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:58.864103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 142/267] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:39:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:39:58.870894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830e094-9f05-4e49-9c57-6c0a67489203 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:59.064653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830e094-9f05-4e49-9c57-6c0a67489203 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:59.107319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb572f-640b-4fec-ba09-daa1ff1bd448 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.107974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb572f-640b-4fec-ba09-daa1ff1bd448 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.108513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb572f-640b-4fec-ba09-daa1ff1bd448 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.111915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 143/268] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:39:59.116028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07197c-08c3-4a07-8dc8-81497d19a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:59.116653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07197c-08c3-4a07-8dc8-81497d19a020 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.116935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07197c-08c3-4a07-8dc8-81497d19a020 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.117211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07197c-08c3-4a07-8dc8-81497d19a020 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.118190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e4b59-e893-4fc8-8239-c0c80cc428d2 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.118536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07197c-08c3-4a07-8dc8-81497d19a020 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:59.118939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e4b59-e893-4fc8-8239-c0c80cc428d2 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.119568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e4b59-e893-4fc8-8239-c0c80cc428d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.120037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07197c-08c3-4a07-8dc8-81497d19a020 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:59.122441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 126/269] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:39:58 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:59.130654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07197c-08c3-4a07-8dc8-81497d19a020 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:59.158875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-0b3e4b59-e893-4fc8-8239-c0c80cc428d2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:59.160715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-0b3e4b59-e893-4fc8-8239-c0c80cc428d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:59.175713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:59.261452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 144/270] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:39:59.262902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc357ca-ad0b-43b2-9c34-d7322a343f8f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.263277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc357ca-ad0b-43b2-9c34-d7322a343f8f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.263552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc357ca-ad0b-43b2-9c34-d7322a343f8f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.264928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 145/271] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:59.270612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955368e4-4d40-4199-938c-ccec8ad795f2 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.270802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07197c-08c3-4a07-8dc8-81497d19a020 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:59.270907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955368e4-4d40-4199-938c-ccec8ad795f2 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.271178 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955368e4-4d40-4199-938c-ccec8ad795f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.301853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-955368e4-4d40-4199-938c-ccec8ad795f2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:59.303091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-955368e4-4d40-4199-938c-ccec8ad795f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:59.327419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cee415-0268-476a-b244-fe9c5d730c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:59.328087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cee415-0268-476a-b244-fe9c5d730c89 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.328432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cee415-0268-476a-b244-fe9c5d730c89 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.328746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cee415-0268-476a-b244-fe9c5d730c89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.329567 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cee415-0268-476a-b244-fe9c5d730c89 None admin] RBAC: Adding query filter params (role.id=4bb04c67143448539c5af8d8bdd621f9, scope.project.id=98f2008c4252437ea79b895753cd0c32, user.id=a257395b62c049b19544830102161cd2) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:59.329875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cee415-0268-476a-b244-fe9c5d730c89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:59.331445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cee415-0268-476a-b244-fe9c5d730c89 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:59.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 127/272] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:39:59 2022] GET /identity/v3/role_assignments?user.id=a257395b62c049b19544830102161cd2&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=4bb04c67143448539c5af8d8bdd621f9 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:39:59.439177 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 146/273] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:39:59.619360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cee415-0268-476a-b244-fe9c5d730c89 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:59.731336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955368e4-4d40-4199-938c-ccec8ad795f2 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:59.741239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:59.748505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cee415-0268-476a-b244-fe9c5d730c89 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:59.771423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:39:59.797032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99041503-b5c0-4d4f-b6f6-65b9175bd112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:59.797834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99041503-b5c0-4d4f-b6f6-65b9175bd112 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.798541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99041503-b5c0-4d4f-b6f6-65b9175bd112 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.798852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99041503-b5c0-4d4f-b6f6-65b9175bd112 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.810221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99041503-b5c0-4d4f-b6f6-65b9175bd112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-99041503-b5c0-4d4f-b6f6-65b9175bd112 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:39:59.815657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:39:59.816787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:39:59.817645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 128/274] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:39:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:39:59.818529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db8465-2f1d-4039-b5a2-45dfa58f08e2 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.818529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db8465-2f1d-4039-b5a2-45dfa58f08e2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.819183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db8465-2f1d-4039-b5a2-45dfa58f08e2 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.820249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 129/275] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:39:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:39:59.824888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db8465-2f1d-4039-b5a2-45dfa58f08e2 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:39:59.868799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955368e4-4d40-4199-938c-ccec8ad795f2 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:59.907907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d85dbf-163f-4652-b788-34fb15a30268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:59.908580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d85dbf-163f-4652-b788-34fb15a30268 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.908862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d85dbf-163f-4652-b788-34fb15a30268 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.909142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d85dbf-163f-4652-b788-34fb15a30268 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.910338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d85dbf-163f-4652-b788-34fb15a30268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:59.911835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d85dbf-163f-4652-b788-34fb15a30268 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:39:59.913620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 147/276] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:39:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:39:59.914634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81851-51eb-4a55-b0cd-73f11d1eaf23 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.914914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81851-51eb-4a55-b0cd-73f11d1eaf23 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.915188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81851-51eb-4a55-b0cd-73f11d1eaf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.936458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-cdb81851-51eb-4a55-b0cd-73f11d1eaf23 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:39:59.937700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cdb81851-51eb-4a55-b0cd-73f11d1eaf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:39:59.944448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db8465-2f1d-4039-b5a2-45dfa58f08e2 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:39:59.981623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d9430-a3f3-4cd3-aec9-3c8310e02969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:39:59.982504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d9430-a3f3-4cd3-aec9-3c8310e02969 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:39:59.982830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d9430-a3f3-4cd3-aec9-3c8310e02969 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:39:59.983111 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d9430-a3f3-4cd3-aec9-3c8310e02969 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:39:59.983778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d9430-a3f3-4cd3-aec9-3c8310e02969 None admin] RBAC: Adding query filter params (name=member) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:39:59.984081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d9430-a3f3-4cd3-aec9-3c8310e02969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:39:59.985507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d9430-a3f3-4cd3-aec9-3c8310e02969 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:00.000223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 130/277] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:39:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:00.000223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d9430-a3f3-4cd3-aec9-3c8310e02969 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:00.015277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 148/278] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:00.016585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb81851-51eb-4a55-b0cd-73f11d1eaf23 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:00.030835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:00.052536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:00.134885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb81851-51eb-4a55-b0cd-73f11d1eaf23 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:00.140770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d9430-a3f3-4cd3-aec9-3c8310e02969 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:00.174095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722309e-718a-4b22-a9b1-15efb7cc5e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:00.175218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722309e-718a-4b22-a9b1-15efb7cc5e32 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:00.175707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722309e-718a-4b22-a9b1-15efb7cc5e32 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:00.176143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722309e-718a-4b22-a9b1-15efb7cc5e32 None admin] PATH_INFO: `/v3/users/4befa81fe1d3462eb10afe3fa4213a28` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:00.181548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abfc66-4704-4255-b875-80e850ea9c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:00.182263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abfc66-4704-4255-b875-80e850ea9c48 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:00.182547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abfc66-4704-4255-b875-80e850ea9c48 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:00.182869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abfc66-4704-4255-b875-80e850ea9c48 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:00.183533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abfc66-4704-4255-b875-80e850ea9c48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:00.185551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abfc66-4704-4255-b875-80e850ea9c48 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:00.193362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722309e-718a-4b22-a9b1-15efb7cc5e32 None admin] RBAC: Authorizing `identity:get_user(user_id=4befa81fe1d3462eb10afe3fa4213a28)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:00.195264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722309e-718a-4b22-a9b1-15efb7cc5e32 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:00.197654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 149/279] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:40:00 2022] GET /identity/v3/users/4befa81fe1d3462eb10afe3fa4213a28 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:00.203847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722309e-718a-4b22-a9b1-15efb7cc5e32 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:00.281915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 131/280] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:39:59 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:40:00.307671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:00.328402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abfc66-4704-4255-b875-80e850ea9c48 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:00.329464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722309e-718a-4b22-a9b1-15efb7cc5e32 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:00.375756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a112a-bc28-4c06-8d11-eabc0e693945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:00.376395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a112a-bc28-4c06-8d11-eabc0e693945 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:00.376743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a112a-bc28-4c06-8d11-eabc0e693945 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:00.377019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a112a-bc28-4c06-8d11-eabc0e693945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:00.379256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a112a-bc28-4c06-8d11-eabc0e693945 None admin] RBAC: Adding query filter params (role.id=3f0c34a1f93e49adb1757f315333cc48, user.id=4befa81fe1d3462eb10afe3fa4213a28, scope.system=all) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:00.379574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a112a-bc28-4c06-8d11-eabc0e693945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:00.381075 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a112a-bc28-4c06-8d11-eabc0e693945 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:00.388185 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 150/281] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 4 11:40:00 2022] GET /identity/v3/role_assignments?user.id=4befa81fe1d3462eb10afe3fa4213a28&scope.system=all&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:00.441089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:00.474743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abfc66-4704-4255-b875-80e850ea9c48 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:00.524156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453be06e-d570-4d68-a124-93ae538430af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:00.524842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453be06e-d570-4d68-a124-93ae538430af None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:00.525225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453be06e-d570-4d68-a124-93ae538430af None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:00.525505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453be06e-d570-4d68-a124-93ae538430af None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:00.538445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453be06e-d570-4d68-a124-93ae538430af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-453be06e-d570-4d68-a124-93ae538430af None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:00.544474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:00.545432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:00.546242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 132/282] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:00.552057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a112a-bc28-4c06-8d11-eabc0e693945 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:00.675059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a112a-bc28-4c06-8d11-eabc0e693945 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:00.719877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d2d85-1a26-45cb-a65e-8f052fead57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:00.720584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d2d85-1a26-45cb-a65e-8f052fead57e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:00.720869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d2d85-1a26-45cb-a65e-8f052fead57e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:00.721150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d2d85-1a26-45cb-a65e-8f052fead57e None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:00.721833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d2d85-1a26-45cb-a65e-8f052fead57e None admin] RBAC: Adding query filter params (name=reader) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:00.722140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d2d85-1a26-45cb-a65e-8f052fead57e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:00.723557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d2d85-1a26-45cb-a65e-8f052fead57e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:00.733901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 151/283] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:00.739561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453be06e-d570-4d68-a124-93ae538430af None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:00.787305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5758877-2279-4e98-b7df-e3579b05d81f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:00.788500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5758877-2279-4e98-b7df-e3579b05d81f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:00.788937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5758877-2279-4e98-b7df-e3579b05d81f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:00.790904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 152/284] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:00.797063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0881b12-1c59-4911-9f10-87b4097e9f33 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:00.797588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0881b12-1c59-4911-9f10-87b4097e9f33 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:00.798008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0881b12-1c59-4911-9f10-87b4097e9f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:00.821425 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f0881b12-1c59-4911-9f10-87b4097e9f33 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:00.822621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f0881b12-1c59-4911-9f10-87b4097e9f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:00.826518 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:00.933578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 153/285] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:00.965951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453be06e-d570-4d68-a124-93ae538430af None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:01.019662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb49c9-3662-401e-95dd-f9fc538c3e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:01.020685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb49c9-3662-401e-95dd-f9fc538c3e04 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:01.021182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb49c9-3662-401e-95dd-f9fc538c3e04 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:01.021673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb49c9-3662-401e-95dd-f9fc538c3e04 None admin] PATH_INFO: `/v3/users/acfd310ad89b4fcaa36546c7df9c8319` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:01.028929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb49c9-3662-401e-95dd-f9fc538c3e04 None admin] RBAC: Authorizing `identity:get_user(user_id=acfd310ad89b4fcaa36546c7df9c8319)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:01.030772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb49c9-3662-401e-95dd-f9fc538c3e04 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:01.032715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 133/286] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:40:00 2022] GET /identity/v3/users/acfd310ad89b4fcaa36546c7df9c8319 => generated 297 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:01.038424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0881b12-1c59-4911-9f10-87b4097e9f33 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:01.163930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0881b12-1c59-4911-9f10-87b4097e9f33 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:01.223915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d483d00-69c8-49fb-bee4-8ce19ad20b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:01.224594 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d483d00-69c8-49fb-bee4-8ce19ad20b79 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:01.224880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d483d00-69c8-49fb-bee4-8ce19ad20b79 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:01.225154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d483d00-69c8-49fb-bee4-8ce19ad20b79 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:01.226317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d483d00-69c8-49fb-bee4-8ce19ad20b79 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:01.227782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d483d00-69c8-49fb-bee4-8ce19ad20b79 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:01.229927 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 154/287] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:01 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:01.236497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb49c9-3662-401e-95dd-f9fc538c3e04 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:01.243047 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:01.260747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d483d00-69c8-49fb-bee4-8ce19ad20b79 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:01.268736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:01.450168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb49c9-3662-401e-95dd-f9fc538c3e04 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:01.454311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d483d00-69c8-49fb-bee4-8ce19ad20b79 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:01.515921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3392df-8816-4206-92f9-48b7ed5b4e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:01.517140 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8810a18-3970-49c1-a8c3-26607d0325f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:01.517824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8810a18-3970-49c1-a8c3-26607d0325f1 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:01.518106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8810a18-3970-49c1-a8c3-26607d0325f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:01.518383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8810a18-3970-49c1-a8c3-26607d0325f1 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32/users/acfd310ad89b4fcaa36546c7df9c8319/roles/7ae225f5f02c4cefaeb687c582acfcca` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:01.519815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3392df-8816-4206-92f9-48b7ed5b4e12 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:01.520092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3392df-8816-4206-92f9-48b7ed5b4e12 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:01.520383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3392df-8816-4206-92f9-48b7ed5b4e12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:01.523608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8810a18-3970-49c1-a8c3-26607d0325f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=98f2008c4252437ea79b895753cd0c32, user_id=acfd310ad89b4fcaa36546c7df9c8319, role_id=7ae225f5f02c4cefaeb687c582acfcca)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:01.525108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8810a18-3970-49c1-a8c3-26607d0325f1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:01.538596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3392df-8816-4206-92f9-48b7ed5b4e12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-7a3392df-8816-4206-92f9-48b7ed5b4e12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:01.544025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:01.548559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:01.549472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 155/288] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:01.551442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3392df-8816-4206-92f9-48b7ed5b4e12 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:01.608365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 134/289] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:40:01 2022] PUT /identity/v3/projects/98f2008c4252437ea79b895753cd0c32/users/acfd310ad89b4fcaa36546c7df9c8319/roles/7ae225f5f02c4cefaeb687c582acfcca => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:01.629691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:01.675172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3392df-8816-4206-92f9-48b7ed5b4e12 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:01.724215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb676a8-7c89-4a1d-bd3e-492a88a6128a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:01.724771 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb676a8-7c89-4a1d-bd3e-492a88a6128a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:01.725053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb676a8-7c89-4a1d-bd3e-492a88a6128a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:01.725329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb676a8-7c89-4a1d-bd3e-492a88a6128a None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:01.726036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb676a8-7c89-4a1d-bd3e-492a88a6128a None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:01.726383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb676a8-7c89-4a1d-bd3e-492a88a6128a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:01.727841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb676a8-7c89-4a1d-bd3e-492a88a6128a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:01.744747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 156/290] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:01.753417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8810a18-3970-49c1-a8c3-26607d0325f1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:01.852399 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:01.908851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8810a18-3970-49c1-a8c3-26607d0325f1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:01.953689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917a3db-916b-43a5-aec9-88b81f7284d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:01.954238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917a3db-916b-43a5-aec9-88b81f7284d3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:01.954510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917a3db-916b-43a5-aec9-88b81f7284d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:01.954783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917a3db-916b-43a5-aec9-88b81f7284d3 None admin] PATH_INFO: `/v3/users/c4a3748f073540d68452a90d491aea72` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:01.955935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917a3db-916b-43a5-aec9-88b81f7284d3 None admin] RBAC: Authorizing `identity:get_user(user_id=c4a3748f073540d68452a90d491aea72)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:01.957479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917a3db-916b-43a5-aec9-88b81f7284d3 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:01.959269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 135/291] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:40:01 2022] GET /identity/v3/users/c4a3748f073540d68452a90d491aea72 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:01.965744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb676a8-7c89-4a1d-bd3e-492a88a6128a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:02.090712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb676a8-7c89-4a1d-bd3e-492a88a6128a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:02.128230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:02.140117 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde9765-2143-4aca-8729-7f0878ed4c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:02.141118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde9765-2143-4aca-8729-7f0878ed4c09 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:02.144768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde9765-2143-4aca-8729-7f0878ed4c09 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:02.144768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde9765-2143-4aca-8729-7f0878ed4c09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:02.263576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde9765-2143-4aca-8729-7f0878ed4c09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:02.265082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde9765-2143-4aca-8729-7f0878ed4c09 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:02.389172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 157/292] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:02.397259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917a3db-916b-43a5-aec9-88b81f7284d3 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:02.540217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917a3db-916b-43a5-aec9-88b81f7284d3 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:02.573153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:02.579358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d7686-9a7c-44c5-95fa-7163715e7f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:02.580049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d7686-9a7c-44c5-95fa-7163715e7f45 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:02.580356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d7686-9a7c-44c5-95fa-7163715e7f45 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:02.585201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d7686-9a7c-44c5-95fa-7163715e7f45 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:02.585201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d7686-9a7c-44c5-95fa-7163715e7f45 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:02.585201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d7686-9a7c-44c5-95fa-7163715e7f45 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:02.585201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 136/293] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:02 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:02.591383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde9765-2143-4aca-8729-7f0878ed4c09 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:02.646661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:02.749387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde9765-2143-4aca-8729-7f0878ed4c09 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:02.798230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0faf6-db20-4665-aea3-05d8e662b7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:02.798835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0faf6-db20-4665-aea3-05d8e662b7da None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:02.799118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0faf6-db20-4665-aea3-05d8e662b7da None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:02.799383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0faf6-db20-4665-aea3-05d8e662b7da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:02.800195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0faf6-db20-4665-aea3-05d8e662b7da None admin] RBAC: Adding query filter params (role.id=3f0c34a1f93e49adb1757f315333cc48, scope.project.id=98f2008c4252437ea79b895753cd0c32, user.id=c4a3748f073540d68452a90d491aea72) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:02.800513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0faf6-db20-4665-aea3-05d8e662b7da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:02.802758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0faf6-db20-4665-aea3-05d8e662b7da None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:02.813538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 158/294] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:02 2022] GET /identity/v3/role_assignments?user.id=c4a3748f073540d68452a90d491aea72&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:03.052133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13420b50-8a6b-40ee-919a-5ac42786daec None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:03.052831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13420b50-8a6b-40ee-919a-5ac42786daec None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:03.053329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13420b50-8a6b-40ee-919a-5ac42786daec None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:03.055196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 137/295] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:03.061122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a648d2-55a0-4487-ae08-c909fead3717 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:03.061421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a648d2-55a0-4487-ae08-c909fead3717 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:03.061672 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a648d2-55a0-4487-ae08-c909fead3717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:03.089384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e0a648d2-55a0-4487-ae08-c909fead3717 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:03.091164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e0a648d2-55a0-4487-ae08-c909fead3717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:03.201218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 159/296] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:03.275053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54993a-59c1-4910-8386-d32ae459638e None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:03.275711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54993a-59c1-4910-8386-d32ae459638e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:03.276210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54993a-59c1-4910-8386-d32ae459638e None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:03.277875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 138/297] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:03.283728 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91cf08-2b76-4a41-9f05-3ddbd6371e4f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:03.284022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91cf08-2b76-4a41-9f05-3ddbd6371e4f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:03.284321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91cf08-2b76-4a41-9f05-3ddbd6371e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:03.307653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-3c91cf08-2b76-4a41-9f05-3ddbd6371e4f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:03.308873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3c91cf08-2b76-4a41-9f05-3ddbd6371e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:03.335249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:03.421749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 160/298] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:03.523062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54993a-59c1-4910-8386-d32ae459638e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:03.573973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92878cab-be87-441f-b1cf-5932a7f3679b None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:03.574192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92878cab-be87-441f-b1cf-5932a7f3679b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:03.574192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92878cab-be87-441f-b1cf-5932a7f3679b None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:03.574192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 161/299] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:40:03.582019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c8ef9-d794-4ec0-9ed4-6530a82fc36a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:03.582392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c8ef9-d794-4ec0-9ed4-6530a82fc36a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:03.582626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c8ef9-d794-4ec0-9ed4-6530a82fc36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:03.612033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-cf7c8ef9-d794-4ec0-9ed4-6530a82fc36a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:03.614566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cf7c8ef9-d794-4ec0-9ed4-6530a82fc36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:03.671706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54993a-59c1-4910-8386-d32ae459638e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:03.710799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 162/300] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:03.722884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b351-89c2-4aaa-b17a-a49c70de9bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:03.723546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b351-89c2-4aaa-b17a-a49c70de9bc8 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:03.723828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b351-89c2-4aaa-b17a-a49c70de9bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:03.724102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b351-89c2-4aaa-b17a-a49c70de9bc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:03.737121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7b351-89c2-4aaa-b17a-a49c70de9bc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-42a7b351-89c2-4aaa-b17a-a49c70de9bc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:03.742607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:03.743579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:03.744392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 139/301] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:03.751603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c8ef9-d794-4ec0-9ed4-6530a82fc36a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:03.762591 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b351-89c2-4aaa-b17a-a49c70de9bc8 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:03.837427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:03.854835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:03.872819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c8ef9-d794-4ec0-9ed4-6530a82fc36a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:03.884612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b351-89c2-4aaa-b17a-a49c70de9bc8 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:03.921061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59949-6e4e-443b-950f-8e43a505bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:03.921717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59949-6e4e-443b-950f-8e43a505bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:03.922007 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59949-6e4e-443b-950f-8e43a505bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:03.922372 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59949-6e4e-443b-950f-8e43a505bbf2 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:03.923080 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c59949-6e4e-443b-950f-8e43a505bbf2 None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:03.923371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c59949-6e4e-443b-950f-8e43a505bbf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:03.924835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c59949-6e4e-443b-950f-8e43a505bbf2 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:03.927604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221dd8-eec7-46ee-acf5-c2357b0fb60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:03.928773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221dd8-eec7-46ee-acf5-c2357b0fb60b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:03.929304 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221dd8-eec7-46ee-acf5-c2357b0fb60b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:03.929784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221dd8-eec7-46ee-acf5-c2357b0fb60b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:03.935723 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 163/302] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:03.941355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59949-6e4e-443b-950f-8e43a505bbf2 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:03.947639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df221dd8-eec7-46ee-acf5-c2357b0fb60b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-df221dd8-eec7-46ee-acf5-c2357b0fb60b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:03.954345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:03.955306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:40:03.956127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:03.963551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 140/303] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:03.964916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221dd8-eec7-46ee-acf5-c2357b0fb60b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:04.058460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59949-6e4e-443b-950f-8e43a505bbf2 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:04.078550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221dd8-eec7-46ee-acf5-c2357b0fb60b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:04.096432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fd74a-f541-450d-8b4d-a1eaea350230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:04.097125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fd74a-f541-450d-8b4d-a1eaea350230 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:04.097549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fd74a-f541-450d-8b4d-a1eaea350230 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:04.097678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:04.097760 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fd74a-f541-450d-8b4d-a1eaea350230 None admin] PATH_INFO: `/v3/users/4befa81fe1d3462eb10afe3fa4213a28` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:04.098888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7fd74a-f541-450d-8b4d-a1eaea350230 None admin] RBAC: Authorizing `identity:get_user(user_id=4befa81fe1d3462eb10afe3fa4213a28)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:04.100365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7fd74a-f541-450d-8b4d-a1eaea350230 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:04.102333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 164/304] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:40:03 2022] GET /identity/v3/users/4befa81fe1d3462eb10afe3fa4213a28 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:04.107580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fd74a-f541-450d-8b4d-a1eaea350230 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:04.121516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f3ea7-72cf-4186-9922-fe52cbb45d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:04.122113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:04.122240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f3ea7-72cf-4186-9922-fe52cbb45d69 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:04.122403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f3ea7-72cf-4186-9922-fe52cbb45d69 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:04.122718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f3ea7-72cf-4186-9922-fe52cbb45d69 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:04.123390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f3ea7-72cf-4186-9922-fe52cbb45d69 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:04.123699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f3ea7-72cf-4186-9922-fe52cbb45d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:04.125128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f3ea7-72cf-4186-9922-fe52cbb45d69 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:04.134201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 141/305] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:04.135028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f3ea7-72cf-4186-9922-fe52cbb45d69 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:04.220400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fd74a-f541-450d-8b4d-a1eaea350230 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:04.256031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f3ea7-72cf-4186-9922-fe52cbb45d69 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:04.267850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeea1e-5e12-414b-ada3-4d5046ab51da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:04.268622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeea1e-5e12-414b-ada3-4d5046ab51da None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:04.268910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeea1e-5e12-414b-ada3-4d5046ab51da None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:04.269185 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeea1e-5e12-414b-ada3-4d5046ab51da None admin] PATH_INFO: `/v3/domains/default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:04.270467 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adeea1e-5e12-414b-ada3-4d5046ab51da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:04.271946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adeea1e-5e12-414b-ada3-4d5046ab51da None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:04.273632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 165/306] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:04.274421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeea1e-5e12-414b-ada3-4d5046ab51da None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:04.304432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc41ea5-dfa2-453d-8b3d-645e505eef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:04.305093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc41ea5-dfa2-453d-8b3d-645e505eef20 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:04.305400 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc41ea5-dfa2-453d-8b3d-645e505eef20 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:04.305677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc41ea5-dfa2-453d-8b3d-645e505eef20 None admin] PATH_INFO: `/v3/system/users/4befa81fe1d3462eb10afe3fa4213a28/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:04.307574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc41ea5-dfa2-453d-8b3d-645e505eef20 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4befa81fe1d3462eb10afe3fa4213a28, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:04.309569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc41ea5-dfa2-453d-8b3d-645e505eef20 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:04.388299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 142/307] 10.222.0.15 () {58 vars in 1334 bytes} [Tue Oct 4 11:40:04 2022] PUT /identity/v3/system/users/4befa81fe1d3462eb10afe3fa4213a28/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:04.388299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc41ea5-dfa2-453d-8b3d-645e505eef20 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:04.391716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:04.421603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeea1e-5e12-414b-ada3-4d5046ab51da None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:04.452191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:04.477535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3cc68-d398-46a8-9c26-aed94f0f88b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:04.478816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3cc68-d398-46a8-9c26-aed94f0f88b8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:04.479307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3cc68-d398-46a8-9c26-aed94f0f88b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:04.479952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3cc68-d398-46a8-9c26-aed94f0f88b8 None admin] PATH_INFO: `/v3/users/9c3549e7281047d29f17acbf5751d8d7` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:04.499494 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3cc68-d398-46a8-9c26-aed94f0f88b8 None admin] RBAC: Authorizing `identity:get_user(user_id=9c3549e7281047d29f17acbf5751d8d7)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:04.501008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3cc68-d398-46a8-9c26-aed94f0f88b8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:04.507956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 166/308] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:40:04 2022] GET /identity/v3/users/9c3549e7281047d29f17acbf5751d8d7 => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:04.509521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b1b50-7322-45be-8979-28f17f5ac405 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:04.510229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b1b50-7322-45be-8979-28f17f5ac405 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:04.510229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b1b50-7322-45be-8979-28f17f5ac405 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:04.511754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 167/309] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:04.512465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b1b50-7322-45be-8979-28f17f5ac405 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:04.518284 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc41ea5-dfa2-453d-8b3d-645e505eef20 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:04.565121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1acd44-4dd0-4b55-9b9e-3e33726d24d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:04.565805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1acd44-4dd0-4b55-9b9e-3e33726d24d4 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:04.566103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1acd44-4dd0-4b55-9b9e-3e33726d24d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:04.566322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1acd44-4dd0-4b55-9b9e-3e33726d24d4 None admin] PATH_INFO: `/v3/groups` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:04.567640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1acd44-4dd0-4b55-9b9e-3e33726d24d4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:04.569633 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1acd44-4dd0-4b55-9b9e-3e33726d24d4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:04.631459 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 143/310] 10.222.0.15 () {60 vars in 1134 bytes} [Tue Oct 4 11:40:04 2022] POST /identity/v3/groups => generated 229 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:40:04.632146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b1b50-7322-45be-8979-28f17f5ac405 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:04.632744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cf064-0156-4074-9c66-65e118067ae3 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:04.633031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cf064-0156-4074-9c66-65e118067ae3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:04.633372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cf064-0156-4074-9c66-65e118067ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:04.655467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-654cf064-0156-4074-9c66-65e118067ae3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:04.656689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-654cf064-0156-4074-9c66-65e118067ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:04.680071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf853d3-8305-4479-8415-296c1ef6860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:04.681270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf853d3-8305-4479-8415-296c1ef6860e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:04.681819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf853d3-8305-4479-8415-296c1ef6860e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:04.682399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf853d3-8305-4479-8415-296c1ef6860e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:04.684465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf853d3-8305-4479-8415-296c1ef6860e None admin] RBAC: Adding query filter params (role.id=7ae225f5f02c4cefaeb687c582acfcca, user.id=9c3549e7281047d29f17acbf5751d8d7, scope.system=all) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:04.685154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf853d3-8305-4479-8415-296c1ef6860e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:04.688153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf853d3-8305-4479-8415-296c1ef6860e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:04.702374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 168/311] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 4 11:40:04 2022] GET /identity/v3/role_assignments?user.id=9c3549e7281047d29f17acbf5751d8d7&scope.system=all&role.id=7ae225f5f02c4cefaeb687c582acfcca => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:04.797565 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 144/312] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:05.068444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf853d3-8305-4479-8415-296c1ef6860e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:05.074086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:05.188359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf853d3-8305-4479-8415-296c1ef6860e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:05.275751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f0d28-9b2e-408a-9849-64db0af6c255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:05.276882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f0d28-9b2e-408a-9849-64db0af6c255 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:05.277830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f0d28-9b2e-408a-9849-64db0af6c255 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:05.278298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f0d28-9b2e-408a-9849-64db0af6c255 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:05.296586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f0d28-9b2e-408a-9849-64db0af6c255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-0a5f0d28-9b2e-408a-9849-64db0af6c255 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:05.303502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:05.304490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:05.305298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 169/313] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:05.315470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cf064-0156-4074-9c66-65e118067ae3 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:05.379734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:05.489360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cf064-0156-4074-9c66-65e118067ae3 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:05.530855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd4a35-e12a-48cd-a338-14b591d780d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:05.531872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd4a35-e12a-48cd-a338-14b591d780d4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:05.532382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd4a35-e12a-48cd-a338-14b591d780d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:05.532869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd4a35-e12a-48cd-a338-14b591d780d4 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:05.533751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd4a35-e12a-48cd-a338-14b591d780d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:05.534256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd4a35-e12a-48cd-a338-14b591d780d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:05.535864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd4a35-e12a-48cd-a338-14b591d780d4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:05.546665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 145/314] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:05.554967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f0d28-9b2e-408a-9849-64db0af6c255 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:05.629908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:05.675347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f0d28-9b2e-408a-9849-64db0af6c255 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:05.715896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84475f21-16dd-4928-8c4a-639f11334a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:05.716583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84475f21-16dd-4928-8c4a-639f11334a2c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:05.716887 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84475f21-16dd-4928-8c4a-639f11334a2c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:05.717144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84475f21-16dd-4928-8c4a-639f11334a2c None admin] PATH_INFO: `/v3/users/acfd310ad89b4fcaa36546c7df9c8319` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:05.718366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84475f21-16dd-4928-8c4a-639f11334a2c None admin] RBAC: Authorizing `identity:get_user(user_id=acfd310ad89b4fcaa36546c7df9c8319)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:05.719835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84475f21-16dd-4928-8c4a-639f11334a2c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:05.721838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 170/315] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:40:05 2022] GET /identity/v3/users/acfd310ad89b4fcaa36546c7df9c8319 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:05.727608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd4a35-e12a-48cd-a338-14b591d780d4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:05.888899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:05.898180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd4a35-e12a-48cd-a338-14b591d780d4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:05.953356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa205bd-f2c3-4217-b33c-e2108e2652a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:05.954246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa205bd-f2c3-4217-b33c-e2108e2652a2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:05.954673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa205bd-f2c3-4217-b33c-e2108e2652a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:05.955095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa205bd-f2c3-4217-b33c-e2108e2652a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:06.047210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa205bd-f2c3-4217-b33c-e2108e2652a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:06.048779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa205bd-f2c3-4217-b33c-e2108e2652a2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:06.167112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:06.175130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 146/316] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:06.180555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84475f21-16dd-4928-8c4a-639f11334a2c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:06.291563 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84475f21-16dd-4928-8c4a-639f11334a2c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:06.348590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:06.351593 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242b38c-923c-4d77-b69a-d3827f83d45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:06.352743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242b38c-923c-4d77-b69a-d3827f83d45a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:06.353498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242b38c-923c-4d77-b69a-d3827f83d45a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:06.353962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242b38c-923c-4d77-b69a-d3827f83d45a None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:06.356854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242b38c-923c-4d77-b69a-d3827f83d45a None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:06.362511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242b38c-923c-4d77-b69a-d3827f83d45a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:06.363515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 171/317] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:06 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:06.373660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa205bd-f2c3-4217-b33c-e2108e2652a2 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:06.502813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa205bd-f2c3-4217-b33c-e2108e2652a2 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:06.548270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d89b41-5256-4416-82b5-4e5ab86dbb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:06.549297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d89b41-5256-4416-82b5-4e5ab86dbb5c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:06.549807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d89b41-5256-4416-82b5-4e5ab86dbb5c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:06.550282 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d89b41-5256-4416-82b5-4e5ab86dbb5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:06.551642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d89b41-5256-4416-82b5-4e5ab86dbb5c None admin] RBAC: Adding query filter params (role.id=7ae225f5f02c4cefaeb687c582acfcca, scope.project.id=98f2008c4252437ea79b895753cd0c32, user.id=acfd310ad89b4fcaa36546c7df9c8319) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:06.552219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d89b41-5256-4416-82b5-4e5ab86dbb5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:06.554965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d89b41-5256-4416-82b5-4e5ab86dbb5c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:06.566848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 147/318] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:06 2022] GET /identity/v3/role_assignments?user.id=acfd310ad89b4fcaa36546c7df9c8319&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=7ae225f5f02c4cefaeb687c582acfcca => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:06.980995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adc7fc-29aa-4e6e-b3ca-3264cd37173d None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:06.981272 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adc7fc-29aa-4e6e-b3ca-3264cd37173d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:06.981545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adc7fc-29aa-4e6e-b3ca-3264cd37173d None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:06.983325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 172/319] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:40:06.989165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e8b26-576c-4ca7-951c-c490f3d0e2c1 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:06.990025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e8b26-576c-4ca7-951c-c490f3d0e2c1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:06.990310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e8b26-576c-4ca7-951c-c490f3d0e2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:07.009224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-3a6e8b26-576c-4ca7-951c-c490f3d0e2c1 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:07.009881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f942db-a250-4067-9d83-2e80b27120cc None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:07.010348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f942db-a250-4067-9d83-2e80b27120cc None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:07.010464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3a6e8b26-576c-4ca7-951c-c490f3d0e2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:07.010841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f942db-a250-4067-9d83-2e80b27120cc None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:07.015592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 173/320] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:07.020279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068d848-edb4-4cf0-8295-1f80b1829d52 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:07.020781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068d848-edb4-4cf0-8295-1f80b1829d52 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:07.021049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068d848-edb4-4cf0-8295-1f80b1829d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:07.045144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1068d848-edb4-4cf0-8295-1f80b1829d52 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:07.046615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1068d848-edb4-4cf0-8295-1f80b1829d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:07.084055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:07.108222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 148/321] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:07.160263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 174/322] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:07.401205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e8b26-576c-4ca7-951c-c490f3d0e2c1 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:07.434633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068d848-edb4-4cf0-8295-1f80b1829d52 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:07.516461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e8b26-576c-4ca7-951c-c490f3d0e2c1 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:07.552724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95740a9-9b85-463a-8898-97974ab636d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:07.553358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95740a9-9b85-463a-8898-97974ab636d1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:07.553633 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95740a9-9b85-463a-8898-97974ab636d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:07.553902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95740a9-9b85-463a-8898-97974ab636d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:07.555015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95740a9-9b85-463a-8898-97974ab636d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:07.556507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95740a9-9b85-463a-8898-97974ab636d1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:07.560213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 149/323] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:07.564269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95740a9-9b85-463a-8898-97974ab636d1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:07.596420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068d848-edb4-4cf0-8295-1f80b1829d52 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:07.638042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:07.650333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:07.665314 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027d05a-3edd-4ba7-beae-047f795ad910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:07.665314 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027d05a-3edd-4ba7-beae-047f795ad910 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:07.665314 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027d05a-3edd-4ba7-beae-047f795ad910 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:07.665314 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027d05a-3edd-4ba7-beae-047f795ad910 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:07.675256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027d05a-3edd-4ba7-beae-047f795ad910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-4027d05a-3edd-4ba7-beae-047f795ad910 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:07.680761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:07.684401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:07.685462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 175/324] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:07.693716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027d05a-3edd-4ba7-beae-047f795ad910 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:07.723490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95740a9-9b85-463a-8898-97974ab636d1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:07.787988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a7993-28ff-4066-8350-0751debe8b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:07.788637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7993-28ff-4066-8350-0751debe8b62 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:07.789027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7993-28ff-4066-8350-0751debe8b62 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:07.789301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7993-28ff-4066-8350-0751debe8b62 None admin] PATH_INFO: `/v3/groups` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:07.789941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7993-28ff-4066-8350-0751debe8b62 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:07.791428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7993-28ff-4066-8350-0751debe8b62 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:07.811613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027d05a-3edd-4ba7-beae-047f795ad910 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:07.851729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901df519-57b9-46ef-b2ce-497d9a278591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:07.852377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901df519-57b9-46ef-b2ce-497d9a278591 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:07.852660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901df519-57b9-46ef-b2ce-497d9a278591 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:07.852934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901df519-57b9-46ef-b2ce-497d9a278591 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:07.853622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901df519-57b9-46ef-b2ce-497d9a278591 None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:07.853926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901df519-57b9-46ef-b2ce-497d9a278591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:07.855359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901df519-57b9-46ef-b2ce-497d9a278591 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:07.865083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 176/325] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:07.866429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc21f43-e266-4111-b7e4-67878c8002a5 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:07.866661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc21f43-e266-4111-b7e4-67878c8002a5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:07.866937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc21f43-e266-4111-b7e4-67878c8002a5 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:07.868259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 177/326] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:07.872564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc21f43-e266-4111-b7e4-67878c8002a5 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:07.883543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 150/327] 10.222.0.15 () {60 vars in 1134 bytes} [Tue Oct 4 11:40:07 2022] POST /identity/v3/groups => generated 226 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:40:07.884929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b12990-bc05-462e-b76d-5e83ddf72a34 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:07.885217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b12990-bc05-462e-b76d-5e83ddf72a34 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:07.889034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b12990-bc05-462e-b76d-5e83ddf72a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:07.918611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-79b12990-bc05-462e-b76d-5e83ddf72a34 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:07.920612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-79b12990-bc05-462e-b76d-5e83ddf72a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:07.969894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:08.020046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 151/328] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:08.050238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc21f43-e266-4111-b7e4-67878c8002a5 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:08.098554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31ec01-3fab-42e1-b072-a224511b9e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:08.099695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31ec01-3fab-42e1-b072-a224511b9e64 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:08.100152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31ec01-3fab-42e1-b072-a224511b9e64 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:08.100655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31ec01-3fab-42e1-b072-a224511b9e64 None admin] PATH_INFO: `/v3/users/4befa81fe1d3462eb10afe3fa4213a28` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:08.102207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31ec01-3fab-42e1-b072-a224511b9e64 None admin] RBAC: Authorizing `identity:get_user(user_id=4befa81fe1d3462eb10afe3fa4213a28)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:08.104330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31ec01-3fab-42e1-b072-a224511b9e64 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:08.107465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 178/329] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:40:07 2022] GET /identity/v3/users/4befa81fe1d3462eb10afe3fa4213a28 => generated 293 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:08.113910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b12990-bc05-462e-b76d-5e83ddf72a34 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:08.176997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:08.246887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b12990-bc05-462e-b76d-5e83ddf72a34 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:08.303654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31ec01-3fab-42e1-b072-a224511b9e64 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:08.312663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478a3a5-d3b1-491e-b56b-89c747ecfea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:08.313691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478a3a5-d3b1-491e-b56b-89c747ecfea5 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:08.314162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478a3a5-d3b1-491e-b56b-89c747ecfea5 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:08.314629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478a3a5-d3b1-491e-b56b-89c747ecfea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:08.316062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478a3a5-d3b1-491e-b56b-89c747ecfea5 None admin] RBAC: Adding query filter params (role.id=3f0c34a1f93e49adb1757f315333cc48, user.id=4befa81fe1d3462eb10afe3fa4213a28, scope.system=all) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:08.316653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478a3a5-d3b1-491e-b56b-89c747ecfea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:08.319513 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478a3a5-d3b1-491e-b56b-89c747ecfea5 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:08.342811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 152/330] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 4 11:40:08 2022] GET /identity/v3/role_assignments?user.id=4befa81fe1d3462eb10afe3fa4213a28&scope.system=all&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:08.488510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31ec01-3fab-42e1-b072-a224511b9e64 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:08.515119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:08.535652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1d2bc-c9f4-4857-a5fa-cfcf3eb9d393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:08.536335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1d2bc-c9f4-4857-a5fa-cfcf3eb9d393 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:08.536614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1d2bc-c9f4-4857-a5fa-cfcf3eb9d393 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:08.536886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1d2bc-c9f4-4857-a5fa-cfcf3eb9d393 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:08.552071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1d2bc-c9f4-4857-a5fa-cfcf3eb9d393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-0ca1d2bc-c9f4-4857-a5fa-cfcf3eb9d393 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:08.557510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:08.559199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:08.560793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 179/331] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:08.564862 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478a3a5-d3b1-491e-b56b-89c747ecfea5 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:08.695713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478a3a5-d3b1-491e-b56b-89c747ecfea5 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:08.706974 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:08.742405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86683585-65c3-420a-bab7-97925e1b7755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:08.743260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86683585-65c3-420a-bab7-97925e1b7755 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:08.743692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86683585-65c3-420a-bab7-97925e1b7755 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:08.743966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86683585-65c3-420a-bab7-97925e1b7755 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:08.744739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86683585-65c3-420a-bab7-97925e1b7755 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:08.747169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86683585-65c3-420a-bab7-97925e1b7755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:08.747719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86683585-65c3-420a-bab7-97925e1b7755 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:08.757816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 153/332] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:08.764036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1d2bc-c9f4-4857-a5fa-cfcf3eb9d393 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:08.918042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1d2bc-c9f4-4857-a5fa-cfcf3eb9d393 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:08.960373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9ce0a-28d0-442f-918c-9f514df90955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:08.961023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9ce0a-28d0-442f-918c-9f514df90955 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:08.961308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9ce0a-28d0-442f-918c-9f514df90955 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:08.961583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9ce0a-28d0-442f-918c-9f514df90955 None admin] PATH_INFO: `/v3/users/9c3549e7281047d29f17acbf5751d8d7` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:08.962715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9ce0a-28d0-442f-918c-9f514df90955 None admin] RBAC: Authorizing `identity:get_user(user_id=9c3549e7281047d29f17acbf5751d8d7)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:08.964186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9ce0a-28d0-442f-918c-9f514df90955 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:08.966703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 180/333] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:40:08 2022] GET /identity/v3/users/9c3549e7281047d29f17acbf5751d8d7 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:08.973344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86683585-65c3-420a-bab7-97925e1b7755 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:09.201096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86683585-65c3-420a-bab7-97925e1b7755 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:09.273063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8f0fc-7fd0-4d49-a5cb-c5e98b05ca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:09.273717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8f0fc-7fd0-4d49-a5cb-c5e98b05ca5a None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:09.273993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8f0fc-7fd0-4d49-a5cb-c5e98b05ca5a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:09.274266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8f0fc-7fd0-4d49-a5cb-c5e98b05ca5a None admin] PATH_INFO: `/v3/system/users/9c3549e7281047d29f17acbf5751d8d7/roles/7ae225f5f02c4cefaeb687c582acfcca` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:09.275751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8f0fc-7fd0-4d49-a5cb-c5e98b05ca5a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9c3549e7281047d29f17acbf5751d8d7, role_id=7ae225f5f02c4cefaeb687c582acfcca)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:09.277188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8f0fc-7fd0-4d49-a5cb-c5e98b05ca5a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:09.345910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 154/334] 10.222.0.15 () {58 vars in 1334 bytes} [Tue Oct 4 11:40:08 2022] PUT /identity/v3/system/users/9c3549e7281047d29f17acbf5751d8d7/roles/7ae225f5f02c4cefaeb687c582acfcca => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:11.227154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfc5d9-a94f-453f-a107-f87166c94ad5 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:11.227696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfc5d9-a94f-453f-a107-f87166c94ad5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:11.228140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfc5d9-a94f-453f-a107-f87166c94ad5 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:11.231004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 181/335] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:11.253248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac23163-bb2c-41a0-8b40-f68559d8cfb2 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:11.253788 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac23163-bb2c-41a0-8b40-f68559d8cfb2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:11.254209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac23163-bb2c-41a0-8b40-f68559d8cfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:11.296459 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7ac23163-bb2c-41a0-8b40-f68559d8cfb2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:11.298634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7ac23163-bb2c-41a0-8b40-f68559d8cfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:11.355082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:11.420396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 155/336] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:12.268273 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfc5d9-a94f-453f-a107-f87166c94ad5 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:12.383344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:12.491820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfc5d9-a94f-453f-a107-f87166c94ad5 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:12.538050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2640e5-f7d9-458d-8e9b-4f9369059a73 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:12.539003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2640e5-f7d9-458d-8e9b-4f9369059a73 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:12.539483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2640e5-f7d9-458d-8e9b-4f9369059a73 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:12.541509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 156/337] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:12.548084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9f197-4d6c-475d-b704-82dc4e3f63d2 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:12.548724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9f197-4d6c-475d-b704-82dc4e3f63d2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:12.549226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9f197-4d6c-475d-b704-82dc4e3f63d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:12.570355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-60d9f197-4d6c-475d-b704-82dc4e3f63d2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:12.571967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-60d9f197-4d6c-475d-b704-82dc4e3f63d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:12.582276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56a69d-47de-407d-ac09-bdf1bc0fcf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:12.583355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56a69d-47de-407d-ac09-bdf1bc0fcf4a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:12.583859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56a69d-47de-407d-ac09-bdf1bc0fcf4a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:12.584384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56a69d-47de-407d-ac09-bdf1bc0fcf4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:12.614714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56a69d-47de-407d-ac09-bdf1bc0fcf4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-5b56a69d-47de-407d-ac09-bdf1bc0fcf4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:12.633692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:12.634664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:12.635473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:12.636090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 182/338] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:12.638333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f7da0-489a-45b5-bc96-8f9dc29c46af None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:12.638838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f7da0-489a-45b5-bc96-8f9dc29c46af None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:12.639337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f7da0-489a-45b5-bc96-8f9dc29c46af None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:12.645958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 183/339] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:12.651868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f7da0-489a-45b5-bc96-8f9dc29c46af None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:12.663568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 157/340] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:12.663568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1406ae-5578-4d6c-b89a-a0c61a0c36f4 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:12.663568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1406ae-5578-4d6c-b89a-a0c61a0c36f4 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:12.663568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1406ae-5578-4d6c-b89a-a0c61a0c36f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:12.689660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-6e1406ae-5578-4d6c-b89a-a0c61a0c36f4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:12.694875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-6e1406ae-5578-4d6c-b89a-a0c61a0c36f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:12.801657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 158/341] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:12.936983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f7da0-489a-45b5-bc96-8f9dc29c46af None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:12.956856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:13.014847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b583fbb-615b-4ab3-8352-43accef9cd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:13.016103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b583fbb-615b-4ab3-8352-43accef9cd9c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:13.016639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b583fbb-615b-4ab3-8352-43accef9cd9c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:13.017132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b583fbb-615b-4ab3-8352-43accef9cd9c None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:13.018360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b583fbb-615b-4ab3-8352-43accef9cd9c None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:13.018902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b583fbb-615b-4ab3-8352-43accef9cd9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:13.021543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b583fbb-615b-4ab3-8352-43accef9cd9c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:13.043793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 184/342] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:13.052926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1406ae-5578-4d6c-b89a-a0c61a0c36f4 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:13.091297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:13.169821 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b583fbb-615b-4ab3-8352-43accef9cd9c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:13.204227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1406ae-5578-4d6c-b89a-a0c61a0c36f4 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:13.249204 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44304efa-4ac4-48b7-bc0a-db8dc2e61f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:13.249842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44304efa-4ac4-48b7-bc0a-db8dc2e61f65 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:13.250123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44304efa-4ac4-48b7-bc0a-db8dc2e61f65 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:13.250394 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44304efa-4ac4-48b7-bc0a-db8dc2e61f65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:13.424302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44304efa-4ac4-48b7-bc0a-db8dc2e61f65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:13.425866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44304efa-4ac4-48b7-bc0a-db8dc2e61f65 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:13.475106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b583fbb-615b-4ab3-8352-43accef9cd9c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:13.486004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:13.570626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6669ad-842a-44a8-8fff-7bd842966e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:13.571680 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6669ad-842a-44a8-8fff-7bd842966e30 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:13.572187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6669ad-842a-44a8-8fff-7bd842966e30 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:13.572664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6669ad-842a-44a8-8fff-7bd842966e30 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:13.604642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6669ad-842a-44a8-8fff-7bd842966e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-6e6669ad-842a-44a8-8fff-7bd842966e30 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:13.616487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:13.618387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:13.620245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:13.624942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 185/343] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:40:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 456 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:13.627484 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0c832-acdb-4f72-9b98-21d1bc44450a None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:13.628235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0c832-acdb-4f72-9b98-21d1bc44450a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:13.628939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0c832-acdb-4f72-9b98-21d1bc44450a None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:13.636322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 186/344] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:13.638019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0c832-acdb-4f72-9b98-21d1bc44450a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:13.679077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 159/345] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:13.679077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44304efa-4ac4-48b7-bc0a-db8dc2e61f65 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:13.699439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:13.833046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0c832-acdb-4f72-9b98-21d1bc44450a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:13.866154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44304efa-4ac4-48b7-bc0a-db8dc2e61f65 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:13.881888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e1bb6-e80b-4da3-b30e-b39e827b5761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:13.882532 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e1bb6-e80b-4da3-b30e-b39e827b5761 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:13.882813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e1bb6-e80b-4da3-b30e-b39e827b5761 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:13.883086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e1bb6-e80b-4da3-b30e-b39e827b5761 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:13.902141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:13.908891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e1bb6-e80b-4da3-b30e-b39e827b5761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:13.915930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0ae06-12cb-4b77-8c78-b969a561c51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-9d0e1bb6-e80b-4da3-b30e-b39e827b5761 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:13.916210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:13.920143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:13.921906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:13.923129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0ae06-12cb-4b77-8c78-b969a561c51c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:13.923129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0ae06-12cb-4b77-8c78-b969a561c51c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:13.923129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0ae06-12cb-4b77-8c78-b969a561c51c None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:13.923471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 187/346] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:13.923593 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0ae06-12cb-4b77-8c78-b969a561c51c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:13.923593 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0ae06-12cb-4b77-8c78-b969a561c51c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:13.923805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b46974-ec4f-44ea-957a-220c6e3c3918 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:13.923943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0ae06-12cb-4b77-8c78-b969a561c51c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:13.924065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b46974-ec4f-44ea-957a-220c6e3c3918 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:13.924065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b46974-ec4f-44ea-957a-220c6e3c3918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:13.934309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 160/347] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:13.936295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24facc-b621-4f6e-b8c7-73082073fb05 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:13.936295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24facc-b621-4f6e-b8c7-73082073fb05 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:13.936295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24facc-b621-4f6e-b8c7-73082073fb05 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:13.939116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 161/348] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:13.939296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24facc-b621-4f6e-b8c7-73082073fb05 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:13.953040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-99b46974-ec4f-44ea-957a-220c6e3c3918 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:13.954295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-99b46974-ec4f-44ea-957a-220c6e3c3918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:14.066793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 188/349] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:14.068796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b46974-ec4f-44ea-957a-220c6e3c3918 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:14.112437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:14.114672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24facc-b621-4f6e-b8c7-73082073fb05 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:14.191875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe0b94-a239-4d54-9994-5e245e93eb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:14.193095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe0b94-a239-4d54-9994-5e245e93eb2c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:14.193684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe0b94-a239-4d54-9994-5e245e93eb2c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:14.194260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe0b94-a239-4d54-9994-5e245e93eb2c None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:14.196495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe0b94-a239-4d54-9994-5e245e93eb2c None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:14.201996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe0b94-a239-4d54-9994-5e245e93eb2c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:14.205289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 162/350] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:13 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:14.206902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe0b94-a239-4d54-9994-5e245e93eb2c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:14.261360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b46974-ec4f-44ea-957a-220c6e3c3918 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:14.303377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:14.305420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ecaf2-f482-407b-b728-cc70113aee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:14.306243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ecaf2-f482-407b-b728-cc70113aee0f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:14.306666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ecaf2-f482-407b-b728-cc70113aee0f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:14.307080 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ecaf2-f482-407b-b728-cc70113aee0f None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:14.307898 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ecaf2-f482-407b-b728-cc70113aee0f None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:14.310114 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ecaf2-f482-407b-b728-cc70113aee0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:14.312270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ecaf2-f482-407b-b728-cc70113aee0f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:14.322973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 189/351] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:14.325251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32779d1d-2f41-4afe-8139-6763293f3ee5 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:14.325690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32779d1d-2f41-4afe-8139-6763293f3ee5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:14.326105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32779d1d-2f41-4afe-8139-6763293f3ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:14.345982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-32779d1d-2f41-4afe-8139-6763293f3ee5 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:14.347635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-32779d1d-2f41-4afe-8139-6763293f3ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:14.414624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe0b94-a239-4d54-9994-5e245e93eb2c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:14.439267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 190/352] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:14.440810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32779d1d-2f41-4afe-8139-6763293f3ee5 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:14.464665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:14.480750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2836df5-5ba2-4962-ad41-af8615b5a946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:14.481325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2836df5-5ba2-4962-ad41-af8615b5a946 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:14.481823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2836df5-5ba2-4962-ad41-af8615b5a946 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:14.482891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2836df5-5ba2-4962-ad41-af8615b5a946 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:14.579464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32779d1d-2f41-4afe-8139-6763293f3ee5 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:14.644712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79ed93-9f24-4b80-9eb9-9b600fc9faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:14.645865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79ed93-9f24-4b80-9eb9-9b600fc9faba None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:14.646305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79ed93-9f24-4b80-9eb9-9b600fc9faba None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:14.646733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79ed93-9f24-4b80-9eb9-9b600fc9faba None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:14.652451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2836df5-5ba2-4962-ad41-af8615b5a946 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:14.654553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2836df5-5ba2-4962-ad41-af8615b5a946 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:14.655683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79ed93-9f24-4b80-9eb9-9b600fc9faba None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:14.657482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79ed93-9f24-4b80-9eb9-9b600fc9faba None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:14.659592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 191/353] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:14 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:14.660814 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79ed93-9f24-4b80-9eb9-9b600fc9faba None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:14.764862 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:14.805845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:14.855969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79ed93-9f24-4b80-9eb9-9b600fc9faba None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:14.880379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 163/354] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:14.881701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2836df5-5ba2-4962-ad41-af8615b5a946 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:14.909497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc8afb-7ee0-4ffa-8633-77f8b4d7b2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:14.910134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8afb-7ee0-4ffa-8633-77f8b4d7b2a3 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:14.910418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8afb-7ee0-4ffa-8633-77f8b4d7b2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:14.910695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8afb-7ee0-4ffa-8633-77f8b4d7b2a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.003331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc8afb-7ee0-4ffa-8633-77f8b4d7b2a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:15.005321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc8afb-7ee0-4ffa-8633-77f8b4d7b2a3 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:15.066785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2836df5-5ba2-4962-ad41-af8615b5a946 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:15.101958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:15.116352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddb4dd-72fc-40c5-aebc-c2feb8a8f371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:15.116987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddb4dd-72fc-40c5-aebc-c2feb8a8f371 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:15.117476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddb4dd-72fc-40c5-aebc-c2feb8a8f371 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:15.117587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeddb4dd-72fc-40c5-aebc-c2feb8a8f371 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.130758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeddb4dd-72fc-40c5-aebc-c2feb8a8f371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:15.132717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 192/355] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:15.132717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc8afb-7ee0-4ffa-8633-77f8b4d7b2a3 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-eeddb4dd-72fc-40c5-aebc-c2feb8a8f371 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:15.135337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:15.136272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:15.137113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 164/356] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:15.137771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddb4dd-72fc-40c5-aebc-c2feb8a8f371 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:15.157420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:15.264334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeddb4dd-72fc-40c5-aebc-c2feb8a8f371 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:15.276865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc8afb-7ee0-4ffa-8633-77f8b4d7b2a3 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:15.310615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd239c8-75aa-4b03-bbfd-634484918917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:15.311388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd239c8-75aa-4b03-bbfd-634484918917 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:15.311673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd239c8-75aa-4b03-bbfd-634484918917 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:15.311955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd239c8-75aa-4b03-bbfd-634484918917 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.313942 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd239c8-75aa-4b03-bbfd-634484918917 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:15.315412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd239c8-75aa-4b03-bbfd-634484918917 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:15.317544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 165/357] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:15 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:15.318894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd239c8-75aa-4b03-bbfd-634484918917 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:15.336831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b0b14-7eb2-4856-9e6c-6c13485eb16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:15.337191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b0b14-7eb2-4856-9e6c-6c13485eb16e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:15.337417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b0b14-7eb2-4856-9e6c-6c13485eb16e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:15.337692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b0b14-7eb2-4856-9e6c-6c13485eb16e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.338812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b0b14-7eb2-4856-9e6c-6c13485eb16e None admin] RBAC: Adding query filter params (group.id=5a82c020be1a496484fae4ad4ba72814, role.id=3f0c34a1f93e49adb1757f315333cc48, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:15.338931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b0b14-7eb2-4856-9e6c-6c13485eb16e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:15.341166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b0b14-7eb2-4856-9e6c-6c13485eb16e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:15.347677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 193/358] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 11:40:15 2022] GET /identity/v3/role_assignments?group.id=5a82c020be1a496484fae4ad4ba72814&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:15.349227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00f6a9-e9d3-40f9-b5d0-0f8079a262e8 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:15.349511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00f6a9-e9d3-40f9-b5d0-0f8079a262e8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:15.349787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00f6a9-e9d3-40f9-b5d0-0f8079a262e8 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.351087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 194/359] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:15.351841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00f6a9-e9d3-40f9-b5d0-0f8079a262e8 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:15.414422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:15.471437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:15.506152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd239c8-75aa-4b03-bbfd-634484918917 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:15.524650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00f6a9-e9d3-40f9-b5d0-0f8079a262e8 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:15.556722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4b130-c358-4564-9d0d-2a4a608570f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:15.557627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4b130-c358-4564-9d0d-2a4a608570f6 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:15.558053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4b130-c358-4564-9d0d-2a4a608570f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:15.558665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4b130-c358-4564-9d0d-2a4a608570f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.573197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4b130-c358-4564-9d0d-2a4a608570f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-5ca4b130-c358-4564-9d0d-2a4a608570f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:15.578984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:15.580421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:15.581637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:15.585063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 166/360] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:40:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:15.586474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4b130-c358-4564-9d0d-2a4a608570f6 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:15.587079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8d9d2-07d4-4a27-8388-d890bc1c1e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:15.587822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8d9d2-07d4-4a27-8388-d890bc1c1e1b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:15.588232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8d9d2-07d4-4a27-8388-d890bc1c1e1b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:15.588616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8d9d2-07d4-4a27-8388-d890bc1c1e1b None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.591657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8d9d2-07d4-4a27-8388-d890bc1c1e1b None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:15.593665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8d9d2-07d4-4a27-8388-d890bc1c1e1b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:15.596504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 195/361] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:15 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:15.597417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8d9d2-07d4-4a27-8388-d890bc1c1e1b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:15.727382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4b130-c358-4564-9d0d-2a4a608570f6 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:15.747049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8d9d2-07d4-4a27-8388-d890bc1c1e1b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:15.758891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:15.785046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398be69-c5d0-4801-abf3-7f69e196bc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:15.786247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398be69-c5d0-4801-abf3-7f69e196bc28 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:15.786678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398be69-c5d0-4801-abf3-7f69e196bc28 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:15.787121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398be69-c5d0-4801-abf3-7f69e196bc28 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.787952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398be69-c5d0-4801-abf3-7f69e196bc28 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:15.789273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398be69-c5d0-4801-abf3-7f69e196bc28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:15.789738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398be69-c5d0-4801-abf3-7f69e196bc28 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:15.798643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c097602-7489-413a-b397-195e4f25ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:15.799482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097602-7489-413a-b397-195e4f25ee23 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:15.799669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097602-7489-413a-b397-195e4f25ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:15.800033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097602-7489-413a-b397-195e4f25ee23 None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.800541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 167/362] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:15.802301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72359237-4835-49bf-ba80-6a665656a087 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:15.802301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72359237-4835-49bf-ba80-6a665656a087 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:15.802301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72359237-4835-49bf-ba80-6a665656a087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.805177 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c097602-7489-413a-b397-195e4f25ee23 None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:15.807143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c097602-7489-413a-b397-195e4f25ee23 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:15.809869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 196/363] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:15 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:15.810372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:15.813253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c097602-7489-413a-b397-195e4f25ee23 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:15.828485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-72359237-4835-49bf-ba80-6a665656a087 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:15.829680 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-72359237-4835-49bf-ba80-6a665656a087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:15.926970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 168/364] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:15.927913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72359237-4835-49bf-ba80-6a665656a087 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:15.947695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c097602-7489-413a-b397-195e4f25ee23 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:15.988193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f83bf-bca8-47b3-8a09-f022ed0df305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:15.989276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f83bf-bca8-47b3-8a09-f022ed0df305 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:15.989845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f83bf-bca8-47b3-8a09-f022ed0df305 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:15.990346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f83bf-bca8-47b3-8a09-f022ed0df305 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:15.991231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f83bf-bca8-47b3-8a09-f022ed0df305 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:15.991752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f83bf-bca8-47b3-8a09-f022ed0df305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:15.993415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f83bf-bca8-47b3-8a09-f022ed0df305 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:16.003285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 197/365] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:16.004593 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f83bf-bca8-47b3-8a09-f022ed0df305 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:16.049075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72359237-4835-49bf-ba80-6a665656a087 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:16.050891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:16.082696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:16.088162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5f31d-3970-4d89-b169-2716c9b5e114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:16.088832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5f31d-3970-4d89-b169-2716c9b5e114 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:16.089107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5f31d-3970-4d89-b169-2716c9b5e114 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:16.089434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5f31d-3970-4d89-b169-2716c9b5e114 None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:16.090612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5f31d-3970-4d89-b169-2716c9b5e114 None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:16.092145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5f31d-3970-4d89-b169-2716c9b5e114 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:16.098084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 169/366] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:15 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:16.099260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5f31d-3970-4d89-b169-2716c9b5e114 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:16.129083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f83bf-bca8-47b3-8a09-f022ed0df305 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:16.169773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b120e1b-5c3e-47e4-8bec-d74d6a789bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:16.170894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b120e1b-5c3e-47e4-8bec-d74d6a789bb2 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:16.171595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b120e1b-5c3e-47e4-8bec-d74d6a789bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:16.172339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b120e1b-5c3e-47e4-8bec-d74d6a789bb2 None admin] PATH_INFO: `/v3/users/9c3549e7281047d29f17acbf5751d8d7` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:16.174529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b120e1b-5c3e-47e4-8bec-d74d6a789bb2 None admin] RBAC: Authorizing `identity:get_user(user_id=9c3549e7281047d29f17acbf5751d8d7)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:16.177490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b120e1b-5c3e-47e4-8bec-d74d6a789bb2 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:16.180825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 198/367] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:40:16 2022] GET /identity/v3/users/9c3549e7281047d29f17acbf5751d8d7 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:16.182558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b120e1b-5c3e-47e4-8bec-d74d6a789bb2 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:16.218841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5f31d-3970-4d89-b169-2716c9b5e114 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:16.263622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cfd08-adbf-4a47-b984-0f44915569a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:16.264274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cfd08-adbf-4a47-b984-0f44915569a1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:16.264557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cfd08-adbf-4a47-b984-0f44915569a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:16.264828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cfd08-adbf-4a47-b984-0f44915569a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:16.265578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cfd08-adbf-4a47-b984-0f44915569a1 None admin] RBAC: Adding query filter params (group.id=5a82c020be1a496484fae4ad4ba72814, role.id=4bb04c67143448539c5af8d8bdd621f9, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:16.265887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cfd08-adbf-4a47-b984-0f44915569a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:16.267365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cfd08-adbf-4a47-b984-0f44915569a1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:16.274774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 170/368] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 11:40:16 2022] GET /identity/v3/role_assignments?group.id=5a82c020be1a496484fae4ad4ba72814&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=4bb04c67143448539c5af8d8bdd621f9 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:16.275667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cfd08-adbf-4a47-b984-0f44915569a1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:16.350341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b120e1b-5c3e-47e4-8bec-d74d6a789bb2 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:16.372423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:16.383240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:16.430417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cfd08-adbf-4a47-b984-0f44915569a1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:16.438307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb382c28-f0fc-4cf1-b28e-4ff9ee975490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:16.439371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb382c28-f0fc-4cf1-b28e-4ff9ee975490 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:16.439865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb382c28-f0fc-4cf1-b28e-4ff9ee975490 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:16.440395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb382c28-f0fc-4cf1-b28e-4ff9ee975490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:16.478356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e4340-2335-4e97-89cf-135647f94d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:16.479086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e4340-2335-4e97-89cf-135647f94d03 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:16.479592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e4340-2335-4e97-89cf-135647f94d03 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:16.480199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e4340-2335-4e97-89cf-135647f94d03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:16.481113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e4340-2335-4e97-89cf-135647f94d03 None admin] RBAC: Adding query filter params (group.id=5a82c020be1a496484fae4ad4ba72814, role.id=3f0c34a1f93e49adb1757f315333cc48, scope.project.id=98f2008c4252437ea79b895753cd0c32) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:16.481604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e4340-2335-4e97-89cf-135647f94d03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:16.483457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e4340-2335-4e97-89cf-135647f94d03 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:16.492653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 171/369] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 11:40:16 2022] GET /identity/v3/role_assignments?group.id=5a82c020be1a496484fae4ad4ba72814&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:16.493977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e4340-2335-4e97-89cf-135647f94d03 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:16.590624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb382c28-f0fc-4cf1-b28e-4ff9ee975490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:16.592142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb382c28-f0fc-4cf1-b28e-4ff9ee975490 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:16.724954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e4340-2335-4e97-89cf-135647f94d03 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:16.788547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:16.810829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8eaa2-1fb9-4f5d-91e1-914524254076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:16.812212 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8eaa2-1fb9-4f5d-91e1-914524254076 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:16.812894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8eaa2-1fb9-4f5d-91e1-914524254076 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:16.813598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8eaa2-1fb9-4f5d-91e1-914524254076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:16.815148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8eaa2-1fb9-4f5d-91e1-914524254076 None admin] RBAC: Adding query filter params (role.id=7ae225f5f02c4cefaeb687c582acfcca, user.id=9c3549e7281047d29f17acbf5751d8d7, scope.system=all) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:16.815907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8eaa2-1fb9-4f5d-91e1-914524254076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:16.819849 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8eaa2-1fb9-4f5d-91e1-914524254076 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:16.845640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 172/370] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Oct 4 11:40:16 2022] GET /identity/v3/role_assignments?user.id=9c3549e7281047d29f17acbf5751d8d7&scope.system=all&role.id=7ae225f5f02c4cefaeb687c582acfcca => generated 510 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:16.847167 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8eaa2-1fb9-4f5d-91e1-914524254076 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:16.875205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 199/371] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:16.882016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb382c28-f0fc-4cf1-b28e-4ff9ee975490 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:16.897399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:17.051084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8eaa2-1fb9-4f5d-91e1-914524254076 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:17.065678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb382c28-f0fc-4cf1-b28e-4ff9ee975490 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:17.114623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9794f83-7138-4d0c-8dda-44dab98cc8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:17.116868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e1449-ef17-467e-b161-52b80aa3be82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:17.117286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794f83-7138-4d0c-8dda-44dab98cc8e8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:17.118135 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e1449-ef17-467e-b161-52b80aa3be82 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:17.118677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794f83-7138-4d0c-8dda-44dab98cc8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:17.119146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794f83-7138-4d0c-8dda-44dab98cc8e8 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:17.120392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e1449-ef17-467e-b161-52b80aa3be82 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:17.121423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e1449-ef17-467e-b161-52b80aa3be82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:17.122334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9794f83-7138-4d0c-8dda-44dab98cc8e8 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:17.124946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9794f83-7138-4d0c-8dda-44dab98cc8e8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:17.132318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 200/372] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:16 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:17.134488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9794f83-7138-4d0c-8dda-44dab98cc8e8 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:17.149187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e1449-ef17-467e-b161-52b80aa3be82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-317e1449-ef17-467e-b161-52b80aa3be82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:17.154703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:17.155672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:17.157129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 173/373] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:40:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:17.161770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e1449-ef17-467e-b161-52b80aa3be82 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:17.220498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:17.236641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:17.248330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9794f83-7138-4d0c-8dda-44dab98cc8e8 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:17.286341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64995d7-004f-4d31-a798-f621c27090a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:17.287393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64995d7-004f-4d31-a798-f621c27090a4 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:17.288006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64995d7-004f-4d31-a798-f621c27090a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:17.288492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64995d7-004f-4d31-a798-f621c27090a4 None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:17.289908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64995d7-004f-4d31-a798-f621c27090a4 None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:17.291583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64995d7-004f-4d31-a798-f621c27090a4 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:17.298031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 201/374] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:17 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:17.300919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64995d7-004f-4d31-a798-f621c27090a4 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:17.302411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e1449-ef17-467e-b161-52b80aa3be82 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:17.351894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e97cad-aaf8-4c1a-bd9e-3cc5b397472a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:17.352541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e97cad-aaf8-4c1a-bd9e-3cc5b397472a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:17.352814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e97cad-aaf8-4c1a-bd9e-3cc5b397472a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:17.353087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e97cad-aaf8-4c1a-bd9e-3cc5b397472a None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:17.353772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e97cad-aaf8-4c1a-bd9e-3cc5b397472a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:17.354076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e97cad-aaf8-4c1a-bd9e-3cc5b397472a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:17.355541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e97cad-aaf8-4c1a-bd9e-3cc5b397472a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:17.376197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 174/375] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:17.383678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e97cad-aaf8-4c1a-bd9e-3cc5b397472a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:17.504841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64995d7-004f-4d31-a798-f621c27090a4 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:17.572131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e97cad-aaf8-4c1a-bd9e-3cc5b397472a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:17.585277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:17.602009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403f627-3129-4f8b-8ad3-5fee2346b2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:17.604740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403f627-3129-4f8b-8ad3-5fee2346b2a3 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:17.605597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403f627-3129-4f8b-8ad3-5fee2346b2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:17.605878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403f627-3129-4f8b-8ad3-5fee2346b2a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:17.606715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7403f627-3129-4f8b-8ad3-5fee2346b2a3 None admin] RBAC: Adding query filter params (group.id=5a82c020be1a496484fae4ad4ba72814, role.id=4bb04c67143448539c5af8d8bdd621f9, scope.project.id=98f2008c4252437ea79b895753cd0c32) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:17.607271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7403f627-3129-4f8b-8ad3-5fee2346b2a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:17.610913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7403f627-3129-4f8b-8ad3-5fee2346b2a3 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:17.615420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 202/376] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 11:40:17 2022] GET /identity/v3/role_assignments?group.id=5a82c020be1a496484fae4ad4ba72814&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=4bb04c67143448539c5af8d8bdd621f9 => generated 260 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:17.637215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db2a2f-b855-42a4-8821-71275b4b8630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:17.637850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2a2f-b855-42a4-8821-71275b4b8630 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:17.638129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2a2f-b855-42a4-8821-71275b4b8630 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:17.638400 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2a2f-b855-42a4-8821-71275b4b8630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:17.740480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db2a2f-b855-42a4-8821-71275b4b8630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:17.742010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db2a2f-b855-42a4-8821-71275b4b8630 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:17.896651 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 175/377] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:17.904596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403f627-3129-4f8b-8ad3-5fee2346b2a3 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:17.920091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:18.056727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403f627-3129-4f8b-8ad3-5fee2346b2a3 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:18.096068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dab671-196f-4493-b38a-c5ddf467146c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:18.096807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dab671-196f-4493-b38a-c5ddf467146c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:18.097103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dab671-196f-4493-b38a-c5ddf467146c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:18.097346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dab671-196f-4493-b38a-c5ddf467146c None admin] PATH_INFO: `/v3/projects/a155f451dd324796b4bda64d212b8b97` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:18.098496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dab671-196f-4493-b38a-c5ddf467146c None admin] RBAC: Authorizing `identity:get_project(project_id=a155f451dd324796b4bda64d212b8b97)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:18.099939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dab671-196f-4493-b38a-c5ddf467146c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:18.101929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 203/378] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:17 2022] GET /identity/v3/projects/a155f451dd324796b4bda64d212b8b97 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:18.107476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db2a2f-b855-42a4-8821-71275b4b8630 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:18.121907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:18.252097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db2a2f-b855-42a4-8821-71275b4b8630 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:18.306245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eabb33-7276-47fe-906d-5d5006f819c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:18.306893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eabb33-7276-47fe-906d-5d5006f819c9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:18.307169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eabb33-7276-47fe-906d-5d5006f819c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:18.307443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eabb33-7276-47fe-906d-5d5006f819c9 None admin] PATH_INFO: `/v3/groups/18499fcfc0314aa5882522f9e8069100` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:18.315168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eabb33-7276-47fe-906d-5d5006f819c9 None admin] RBAC: Authorizing `identity:get_group(group_id=18499fcfc0314aa5882522f9e8069100)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:18.316735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eabb33-7276-47fe-906d-5d5006f819c9 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:18.318511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 176/379] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:18 2022] GET /identity/v3/groups/18499fcfc0314aa5882522f9e8069100 => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:18.324919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dab671-196f-4493-b38a-c5ddf467146c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:18.423129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b419d00-e6f3-4a4a-abac-3126a21986b5 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:18.423446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b419d00-e6f3-4a4a-abac-3126a21986b5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:18.423673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b419d00-e6f3-4a4a-abac-3126a21986b5 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:18.425191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 177/380] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:40:18.428535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:18.431325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4fb38-be00-47d6-9364-3bfa3176f431 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:18.431603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4fb38-be00-47d6-9364-3bfa3176f431 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:18.431876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4fb38-be00-47d6-9364-3bfa3176f431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:18.461331 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-51f4fb38-be00-47d6-9364-3bfa3176f431 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:18.462528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-51f4fb38-be00-47d6-9364-3bfa3176f431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:18.486828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dab671-196f-4493-b38a-c5ddf467146c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:18.551884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841308cc-6e55-4dfb-b354-b9dd4cf8adcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:18.552445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841308cc-6e55-4dfb-b354-b9dd4cf8adcf None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:18.552891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841308cc-6e55-4dfb-b354-b9dd4cf8adcf None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:18.553375 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841308cc-6e55-4dfb-b354-b9dd4cf8adcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:18.554277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841308cc-6e55-4dfb-b354-b9dd4cf8adcf None admin] RBAC: Adding query filter params (group.id=18499fcfc0314aa5882522f9e8069100, role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=a155f451dd324796b4bda64d212b8b97) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:18.554732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841308cc-6e55-4dfb-b354-b9dd4cf8adcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:18.556362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841308cc-6e55-4dfb-b354-b9dd4cf8adcf None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:18.570817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 178/381] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:18.571184 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 204/382] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 11:40:18 2022] GET /identity/v3/role_assignments?group.id=18499fcfc0314aa5882522f9e8069100&scope.project.id=a155f451dd324796b4bda64d212b8b97&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 260 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:18.900138 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4fb38-be00-47d6-9364-3bfa3176f431 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:18.903254 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:19.056677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4fb38-be00-47d6-9364-3bfa3176f431 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:19.114387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ded50-53eb-433e-b91c-dc44a0cb359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:19.115300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ded50-53eb-433e-b91c-dc44a0cb359a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:19.115704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ded50-53eb-433e-b91c-dc44a0cb359a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:19.116115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ded50-53eb-433e-b91c-dc44a0cb359a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:19.137316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ded50-53eb-433e-b91c-dc44a0cb359a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-da0ded50-53eb-433e-b91c-dc44a0cb359a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:19.142636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:19.147820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:19.148661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 179/383] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:19.151259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841308cc-6e55-4dfb-b354-b9dd4cf8adcf None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:19.295400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841308cc-6e55-4dfb-b354-b9dd4cf8adcf None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:19.295926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca072e1-597c-4269-9a04-8ea3c69b54af None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:19.296469 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca072e1-597c-4269-9a04-8ea3c69b54af None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:19.296969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca072e1-597c-4269-9a04-8ea3c69b54af None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:19.302340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 180/384] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:40:19.306986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28533b6-e617-4bc9-a5ce-825b56c87e4c None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:19.307513 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28533b6-e617-4bc9-a5ce-825b56c87e4c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:19.308064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28533b6-e617-4bc9-a5ce-825b56c87e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:19.334099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455a09d-b3aa-455d-b55a-2c391b34e127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:19.334714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455a09d-b3aa-455d-b55a-2c391b34e127 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:19.334995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455a09d-b3aa-455d-b55a-2c391b34e127 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:19.335269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455a09d-b3aa-455d-b55a-2c391b34e127 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:19.335932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455a09d-b3aa-455d-b55a-2c391b34e127 None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:19.336261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455a09d-b3aa-455d-b55a-2c391b34e127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:19.337622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455a09d-b3aa-455d-b55a-2c391b34e127 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:19.350171 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:19.355718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 205/385] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:19.360790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e28533b6-e617-4bc9-a5ce-825b56c87e4c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:19.362215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e28533b6-e617-4bc9-a5ce-825b56c87e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:19.370023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455a09d-b3aa-455d-b55a-2c391b34e127 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:19.480520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455a09d-b3aa-455d-b55a-2c391b34e127 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:19.521946 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:19.524220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef9359-94d2-4699-a5fc-c7c5b97d529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:19.524220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef9359-94d2-4699-a5fc-c7c5b97d529a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:19.524220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef9359-94d2-4699-a5fc-c7c5b97d529a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:19.524220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef9359-94d2-4699-a5fc-c7c5b97d529a None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:19.525847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef9359-94d2-4699-a5fc-c7c5b97d529a None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:19.527337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef9359-94d2-4699-a5fc-c7c5b97d529a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:19.533538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 206/386] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:19 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:19.540739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef9359-94d2-4699-a5fc-c7c5b97d529a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:19.550654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 181/387] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:19.608727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:19.643225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef9359-94d2-4699-a5fc-c7c5b97d529a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:19.684437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecde75-16d1-4df9-bc29-d7e6e294ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:19.685076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecde75-16d1-4df9-bc29-d7e6e294ce5c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:19.685358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecde75-16d1-4df9-bc29-d7e6e294ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:19.685633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecde75-16d1-4df9-bc29-d7e6e294ce5c None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:19.693270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecde75-16d1-4df9-bc29-d7e6e294ce5c None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:19.694908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecde75-16d1-4df9-bc29-d7e6e294ce5c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:19.697127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 207/388] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:19 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:19.704124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28533b6-e617-4bc9-a5ce-825b56c87e4c None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:19.834652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aab44-ea78-4323-bc48-f1b0465a9ef1 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:19.835338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aab44-ea78-4323-bc48-f1b0465a9ef1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:19.835915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aab44-ea78-4323-bc48-f1b0465a9ef1 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:19.837806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 208/389] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:40:19.843820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6a0dc-e7b9-4e23-9d82-ca392bcfce0e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:19.844473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6a0dc-e7b9-4e23-9d82-ca392bcfce0e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:19.844972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6a0dc-e7b9-4e23-9d82-ca392bcfce0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:19.865785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-dcd6a0dc-e7b9-4e23-9d82-ca392bcfce0e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:19.867323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-dcd6a0dc-e7b9-4e23-9d82-ca392bcfce0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:19.885971 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28533b6-e617-4bc9-a5ce-825b56c87e4c None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:19.933095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657fcff-765d-4ffa-af7e-4dceedcadd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:19.934662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657fcff-765d-4ffa-af7e-4dceedcadd13 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:19.935255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657fcff-765d-4ffa-af7e-4dceedcadd13 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:19.935720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657fcff-765d-4ffa-af7e-4dceedcadd13 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/groups/5a82c020be1a496484fae4ad4ba72814/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:19.938020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657fcff-765d-4ffa-af7e-4dceedcadd13 None admin] RBAC: Authorizing `identity:create_grant(project_id=973ebf7f6f414136a6ae5a7d1a1a919c, group_id=5a82c020be1a496484fae4ad4ba72814, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:19.939709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657fcff-765d-4ffa-af7e-4dceedcadd13 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:19.946767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 209/390] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:20.014538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 182/391] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Oct 4 11:40:19 2022] PUT /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/groups/5a82c020be1a496484fae4ad4ba72814/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:20.027199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6a0dc-e7b9-4e23-9d82-ca392bcfce0e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:20.109108 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:20.162881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6a0dc-e7b9-4e23-9d82-ca392bcfce0e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:20.207953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88990d-fbf0-41bb-b26e-f05f5901a663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:20.208630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88990d-fbf0-41bb-b26e-f05f5901a663 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:20.208913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88990d-fbf0-41bb-b26e-f05f5901a663 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:20.209187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88990d-fbf0-41bb-b26e-f05f5901a663 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:20.234968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88990d-fbf0-41bb-b26e-f05f5901a663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-fb88990d-fbf0-41bb-b26e-f05f5901a663 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:20.244224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:20.245162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:20.246492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 210/392] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:40:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:20.253398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657fcff-765d-4ffa-af7e-4dceedcadd13 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:20.263012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88990d-fbf0-41bb-b26e-f05f5901a663 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:20.341551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:20.408229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657fcff-765d-4ffa-af7e-4dceedcadd13 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:20.431543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88990d-fbf0-41bb-b26e-f05f5901a663 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:20.445104 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562da0ff-564c-44c9-9300-47956ffd623b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:20.445886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562da0ff-564c-44c9-9300-47956ffd623b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:20.446012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562da0ff-564c-44c9-9300-47956ffd623b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:20.446243 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562da0ff-564c-44c9-9300-47956ffd623b None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:20.446911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562da0ff-564c-44c9-9300-47956ffd623b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:20.447211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562da0ff-564c-44c9-9300-47956ffd623b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:20.448930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562da0ff-564c-44c9-9300-47956ffd623b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:20.457569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 183/393] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:20.459120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ace49-d208-4d3c-a978-ece1f4844831 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:20.459120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ace49-d208-4d3c-a978-ece1f4844831 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:20.459439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ace49-d208-4d3c-a978-ece1f4844831 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:20.460863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 184/394] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:20.463796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ace49-d208-4d3c-a978-ece1f4844831 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:20.466172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:20.480871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ee07f-f55f-40f2-8d39-2b6767c0b6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:20.481753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ee07f-f55f-40f2-8d39-2b6767c0b6e4 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:20.482385 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ee07f-f55f-40f2-8d39-2b6767c0b6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:20.482713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ee07f-f55f-40f2-8d39-2b6767c0b6e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:20.496022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ee07f-f55f-40f2-8d39-2b6767c0b6e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-cf5ee07f-f55f-40f2-8d39-2b6767c0b6e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:20.502862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:20.503788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:20.504764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 211/395] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:20.505785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393290-d176-42f5-a74b-22e750004f11 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:20.506070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393290-d176-42f5-a74b-22e750004f11 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:20.506449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393290-d176-42f5-a74b-22e750004f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:20.527850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b9393290-d176-42f5-a74b-22e750004f11 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:20.529218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b9393290-d176-42f5-a74b-22e750004f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:20.585148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ace49-d208-4d3c-a978-ece1f4844831 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:20.601881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:20.613183 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 212/396] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:20.620304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393290-d176-42f5-a74b-22e750004f11 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:20.641120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62be95d-2bac-4ea8-8c57-6c068e9eb760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:20.641825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62be95d-2bac-4ea8-8c57-6c068e9eb760 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:20.642108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62be95d-2bac-4ea8-8c57-6c068e9eb760 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:20.642379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62be95d-2bac-4ea8-8c57-6c068e9eb760 None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:20.643808 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62be95d-2bac-4ea8-8c57-6c068e9eb760 None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:20.645416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62be95d-2bac-4ea8-8c57-6c068e9eb760 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:20.651510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 185/397] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:20 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:20.652960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62be95d-2bac-4ea8-8c57-6c068e9eb760 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:20.780247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393290-d176-42f5-a74b-22e750004f11 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:20.805506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62be95d-2bac-4ea8-8c57-6c068e9eb760 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:20.817997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:20.875252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17293c81-8c21-4c06-9394-e2290ad8a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:20.875864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17293c81-8c21-4c06-9394-e2290ad8a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:20.876138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17293c81-8c21-4c06-9394-e2290ad8a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:20.876433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17293c81-8c21-4c06-9394-e2290ad8a8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:20.877102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17293c81-8c21-4c06-9394-e2290ad8a8c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:20.877403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17293c81-8c21-4c06-9394-e2290ad8a8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:20.880299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17293c81-8c21-4c06-9394-e2290ad8a8c7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:20.882546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376224e-dd1d-4bf8-97c6-7277d32c72b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:20.883556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376224e-dd1d-4bf8-97c6-7277d32c72b5 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:20.884048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376224e-dd1d-4bf8-97c6-7277d32c72b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:20.884558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376224e-dd1d-4bf8-97c6-7277d32c72b5 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:20.886149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376224e-dd1d-4bf8-97c6-7277d32c72b5 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:20.887921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376224e-dd1d-4bf8-97c6-7277d32c72b5 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:20.890287 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 186/398] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:20 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:20.895621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 213/399] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:20.897434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17293c81-8c21-4c06-9394-e2290ad8a8c7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:20.898330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376224e-dd1d-4bf8-97c6-7277d32c72b5 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:20.972537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:21.003730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17293c81-8c21-4c06-9394-e2290ad8a8c7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:21.029424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376224e-dd1d-4bf8-97c6-7277d32c72b5 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:21.039408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae69453-85aa-4362-86b2-f8d910ebc516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:21.039959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae69453-85aa-4362-86b2-f8d910ebc516 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:21.040270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae69453-85aa-4362-86b2-f8d910ebc516 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:21.040550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae69453-85aa-4362-86b2-f8d910ebc516 None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:21.041757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae69453-85aa-4362-86b2-f8d910ebc516 None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:21.043211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae69453-85aa-4362-86b2-f8d910ebc516 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:21.045174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 214/400] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:20 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:21.045971 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae69453-85aa-4362-86b2-f8d910ebc516 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:21.070583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec2341-5ce8-4382-bb1c-8e87634a1f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:21.071398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec2341-5ce8-4382-bb1c-8e87634a1f12 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:21.071684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec2341-5ce8-4382-bb1c-8e87634a1f12 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:21.072059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec2341-5ce8-4382-bb1c-8e87634a1f12 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/groups/5a82c020be1a496484fae4ad4ba72814/roles/4bb04c67143448539c5af8d8bdd621f9` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:21.075382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec2341-5ce8-4382-bb1c-8e87634a1f12 None admin] RBAC: Authorizing `identity:create_grant(project_id=973ebf7f6f414136a6ae5a7d1a1a919c, group_id=5a82c020be1a496484fae4ad4ba72814, role_id=4bb04c67143448539c5af8d8bdd621f9)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:21.077519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec2341-5ce8-4382-bb1c-8e87634a1f12 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:21.095334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:21.142185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 187/401] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Oct 4 11:40:20 2022] PUT /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c/groups/5a82c020be1a496484fae4ad4ba72814/roles/4bb04c67143448539c5af8d8bdd621f9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:21.143970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec2341-5ce8-4382-bb1c-8e87634a1f12 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:21.154685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae69453-85aa-4362-86b2-f8d910ebc516 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:21.223066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02517e-6804-4cdd-81b3-eaddee2f78fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:21.223717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02517e-6804-4cdd-81b3-eaddee2f78fd None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:21.224074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02517e-6804-4cdd-81b3-eaddee2f78fd None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:21.224326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02517e-6804-4cdd-81b3-eaddee2f78fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:21.249616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b02517e-6804-4cdd-81b3-eaddee2f78fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-2b02517e-6804-4cdd-81b3-eaddee2f78fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:21.255211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:21.256359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:21.257169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:21.262206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 215/402] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:40:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:21.268595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02517e-6804-4cdd-81b3-eaddee2f78fd None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:21.285936 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec2341-5ce8-4382-bb1c-8e87634a1f12 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:21.301665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:21.332978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8be64e-f302-4609-96d6-03447580e957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:21.333921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8be64e-f302-4609-96d6-03447580e957 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:21.334667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8be64e-f302-4609-96d6-03447580e957 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:21.335284 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8be64e-f302-4609-96d6-03447580e957 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:21.340424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8be64e-f302-4609-96d6-03447580e957 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:21.340424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8be64e-f302-4609-96d6-03447580e957 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:21.340424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 188/403] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:21 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:21.341516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3567f-30bc-46a8-a14f-3178568b689d None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:21.341804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3567f-30bc-46a8-a14f-3178568b689d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:21.342060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3567f-30bc-46a8-a14f-3178568b689d None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:21.348512 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 189/404] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:21.349907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3567f-30bc-46a8-a14f-3178568b689d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:21.425769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02517e-6804-4cdd-81b3-eaddee2f78fd None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:21.436370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:21.474774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fba522-147a-4aac-ae30-63d6122eb676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:21.475757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba522-147a-4aac-ae30-63d6122eb676 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:21.476446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba522-147a-4aac-ae30-63d6122eb676 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:21.477248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba522-147a-4aac-ae30-63d6122eb676 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:21.478504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fba522-147a-4aac-ae30-63d6122eb676 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:21.479029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fba522-147a-4aac-ae30-63d6122eb676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:21.481222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fba522-147a-4aac-ae30-63d6122eb676 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:21.484482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3567f-30bc-46a8-a14f-3178568b689d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:21.492293 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 216/405] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:21.493788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc9c12-fecc-4c1b-aec9-21ef0bf06315 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:21.494278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc9c12-fecc-4c1b-aec9-21ef0bf06315 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:21.494777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc9c12-fecc-4c1b-aec9-21ef0bf06315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:21.515816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-29bc9c12-fecc-4c1b-aec9-21ef0bf06315 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:21.517985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-29bc9c12-fecc-4c1b-aec9-21ef0bf06315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:21.524879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869207b-9ac7-4f8f-be83-46e76cfad407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:21.525641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869207b-9ac7-4f8f-be83-46e76cfad407 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:21.525924 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869207b-9ac7-4f8f-be83-46e76cfad407 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:21.526199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869207b-9ac7-4f8f-be83-46e76cfad407 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32/groups/5a82c020be1a496484fae4ad4ba72814/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:21.528358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869207b-9ac7-4f8f-be83-46e76cfad407 None admin] RBAC: Authorizing `identity:create_grant(project_id=98f2008c4252437ea79b895753cd0c32, group_id=5a82c020be1a496484fae4ad4ba72814, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:21.529900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869207b-9ac7-4f8f-be83-46e76cfad407 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:21.593084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 217/406] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:21.593762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc9c12-fecc-4c1b-aec9-21ef0bf06315 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:21.638385 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 190/407] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Oct 4 11:40:21 2022] PUT /identity/v3/projects/98f2008c4252437ea79b895753cd0c32/groups/5a82c020be1a496484fae4ad4ba72814/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:21.715799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:21.720995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc9c12-fecc-4c1b-aec9-21ef0bf06315 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:21.768661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a669bf-3e13-4079-ac42-db18aabf74d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:21.769329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a669bf-3e13-4079-ac42-db18aabf74d0 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:21.769607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a669bf-3e13-4079-ac42-db18aabf74d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:21.769885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a669bf-3e13-4079-ac42-db18aabf74d0 None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:21.771011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a669bf-3e13-4079-ac42-db18aabf74d0 None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:21.772471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a669bf-3e13-4079-ac42-db18aabf74d0 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:21.774023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 218/408] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:21 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:21.784311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869207b-9ac7-4f8f-be83-46e76cfad407 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:21.863246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:21.936180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869207b-9ac7-4f8f-be83-46e76cfad407 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:21.975738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d7dbe-c870-4edb-8fcf-05ffcf889aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:21.976516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d7dbe-c870-4edb-8fcf-05ffcf889aee None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:21.976785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d7dbe-c870-4edb-8fcf-05ffcf889aee None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:21.977063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d7dbe-c870-4edb-8fcf-05ffcf889aee None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:21.978196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d7dbe-c870-4edb-8fcf-05ffcf889aee None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:21.979659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d7dbe-c870-4edb-8fcf-05ffcf889aee None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:21.982421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 191/409] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:21 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:21.988856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a669bf-3e13-4079-ac42-db18aabf74d0 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:22.021006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d7dbe-c870-4edb-8fcf-05ffcf889aee None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:22.117912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a669bf-3e13-4079-ac42-db18aabf74d0 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:22.163178 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36152812-8435-4f59-8d2d-2821035649c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:22.163848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36152812-8435-4f59-8d2d-2821035649c1 None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:22.164124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36152812-8435-4f59-8d2d-2821035649c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:22.164421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36152812-8435-4f59-8d2d-2821035649c1 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32/groups/5a82c020be1a496484fae4ad4ba72814/roles/4bb04c67143448539c5af8d8bdd621f9` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:22.168675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36152812-8435-4f59-8d2d-2821035649c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=98f2008c4252437ea79b895753cd0c32, group_id=5a82c020be1a496484fae4ad4ba72814, role_id=4bb04c67143448539c5af8d8bdd621f9)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:22.170136 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36152812-8435-4f59-8d2d-2821035649c1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:22.237472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 219/410] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Oct 4 11:40:21 2022] PUT /identity/v3/projects/98f2008c4252437ea79b895753cd0c32/groups/5a82c020be1a496484fae4ad4ba72814/roles/4bb04c67143448539c5af8d8bdd621f9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:22.238308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d7dbe-c870-4edb-8fcf-05ffcf889aee None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:22.247631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:22.285791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa09067-7234-4d9d-8257-5bc5d8caa78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:22.286449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa09067-7234-4d9d-8257-5bc5d8caa78f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:22.286731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa09067-7234-4d9d-8257-5bc5d8caa78f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:22.287006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa09067-7234-4d9d-8257-5bc5d8caa78f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:22.307774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa09067-7234-4d9d-8257-5bc5d8caa78f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-8fa09067-7234-4d9d-8257-5bc5d8caa78f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:22.313434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:22.315397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:22.317262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 192/411] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:40:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:22.320606 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36152812-8435-4f59-8d2d-2821035649c1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:22.355642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:22.451585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36152812-8435-4f59-8d2d-2821035649c1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:22.500864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bb18b-b69d-46c5-8cfe-7ee62fbe3ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:22.501486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bb18b-b69d-46c5-8cfe-7ee62fbe3ee0 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:22.501769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bb18b-b69d-46c5-8cfe-7ee62fbe3ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:22.502044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bb18b-b69d-46c5-8cfe-7ee62fbe3ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:22.502740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bb18b-b69d-46c5-8cfe-7ee62fbe3ee0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:22.503048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bb18b-b69d-46c5-8cfe-7ee62fbe3ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:22.504502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bb18b-b69d-46c5-8cfe-7ee62fbe3ee0 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:22.520642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 220/412] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:22.526351 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa09067-7234-4d9d-8257-5bc5d8caa78f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:22.632889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa09067-7234-4d9d-8257-5bc5d8caa78f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:22.669665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f10135-5544-41c0-9142-8f370c871924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:22.670306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f10135-5544-41c0-9142-8f370c871924 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:22.670588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f10135-5544-41c0-9142-8f370c871924 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:22.670862 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f10135-5544-41c0-9142-8f370c871924 None admin] PATH_INFO: `/v3/groups/18499fcfc0314aa5882522f9e8069100` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:22.671987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f10135-5544-41c0-9142-8f370c871924 None admin] RBAC: Authorizing `identity:get_group(group_id=18499fcfc0314aa5882522f9e8069100)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:22.673506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f10135-5544-41c0-9142-8f370c871924 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:22.675403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 193/413] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:22 2022] GET /identity/v3/groups/18499fcfc0314aa5882522f9e8069100 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:22.680567 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bb18b-b69d-46c5-8cfe-7ee62fbe3ee0 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:22.818899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ca572-2dc4-413e-97d2-ffbcf8a5d254 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:22.819390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ca572-2dc4-413e-97d2-ffbcf8a5d254 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:22.819887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ca572-2dc4-413e-97d2-ffbcf8a5d254 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:22.822861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 194/414] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:22.830656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a5cbd-9aa3-43cf-a6cf-6f083cfe7524 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:22.830992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a5cbd-9aa3-43cf-a6cf-6f083cfe7524 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:22.831492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a5cbd-9aa3-43cf-a6cf-6f083cfe7524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:22.848622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:22.863519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bb18b-b69d-46c5-8cfe-7ee62fbe3ee0 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:22.887274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a35a5cbd-9aa3-43cf-a6cf-6f083cfe7524 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:22.891658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a35a5cbd-9aa3-43cf-a6cf-6f083cfe7524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:22.916619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d01655-dee8-4d55-bde8-f2dd638ebab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:22.918197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d01655-dee8-4d55-bde8-f2dd638ebab4 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:22.918844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d01655-dee8-4d55-bde8-f2dd638ebab4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:22.920694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d01655-dee8-4d55-bde8-f2dd638ebab4 None admin] PATH_INFO: `/v3/projects/a155f451dd324796b4bda64d212b8b97` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:22.923076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d01655-dee8-4d55-bde8-f2dd638ebab4 None admin] RBAC: Authorizing `identity:get_project(project_id=a155f451dd324796b4bda64d212b8b97)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:22.924997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d01655-dee8-4d55-bde8-f2dd638ebab4 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:22.927819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 221/415] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:22 2022] GET /identity/v3/projects/a155f451dd324796b4bda64d212b8b97 => generated 344 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:22.935535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d01655-dee8-4d55-bde8-f2dd638ebab4 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:22.953008 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:23.028142 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 195/416] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:23.098218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d01655-dee8-4d55-bde8-f2dd638ebab4 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:23.138390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ed25-7576-4555-87ff-59a4e8e6f33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:23.139092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ed25-7576-4555-87ff-59a4e8e6f33e None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:23.139436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ed25-7576-4555-87ff-59a4e8e6f33e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:23.139719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ed25-7576-4555-87ff-59a4e8e6f33e None admin] PATH_INFO: `/v3/projects/a155f451dd324796b4bda64d212b8b97/groups/18499fcfc0314aa5882522f9e8069100/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:23.142394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67ed25-7576-4555-87ff-59a4e8e6f33e None admin] RBAC: Authorizing `identity:create_grant(project_id=a155f451dd324796b4bda64d212b8b97, group_id=18499fcfc0314aa5882522f9e8069100, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:23.143975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67ed25-7576-4555-87ff-59a4e8e6f33e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:23.294303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 222/417] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Oct 4 11:40:22 2022] PUT /identity/v3/projects/a155f451dd324796b4bda64d212b8b97/groups/18499fcfc0314aa5882522f9e8069100/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:23.344837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a5cbd-9aa3-43cf-a6cf-6f083cfe7524 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:23.487281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a5cbd-9aa3-43cf-a6cf-6f083cfe7524 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:23.536011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ab40d-b1c7-4338-885b-768debd1b74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:23.537020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ab40d-b1c7-4338-885b-768debd1b74a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:23.537547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ab40d-b1c7-4338-885b-768debd1b74a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:23.538002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ab40d-b1c7-4338-885b-768debd1b74a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:23.551990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ab40d-b1c7-4338-885b-768debd1b74a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-142ab40d-b1c7-4338-885b-768debd1b74a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:23.557358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:23.559057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:23.559902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:23.562398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 196/418] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:23.567945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ed25-7576-4555-87ff-59a4e8e6f33e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:23.588837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:23.697759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ed25-7576-4555-87ff-59a4e8e6f33e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:23.756926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07336133-8984-4641-94bf-3f21389bea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:23.757475 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07336133-8984-4641-94bf-3f21389bea50 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:23.757754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07336133-8984-4641-94bf-3f21389bea50 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:23.758029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07336133-8984-4641-94bf-3f21389bea50 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:23.758717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07336133-8984-4641-94bf-3f21389bea50 None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:23.759025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07336133-8984-4641-94bf-3f21389bea50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:23.760461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07336133-8984-4641-94bf-3f21389bea50 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:23.775822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 223/419] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:23.787220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ab40d-b1c7-4338-885b-768debd1b74a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:23.841438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:23.867999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46625d-547e-4807-ad89-fbd0560a5e4f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:23.868560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46625d-547e-4807-ad89-fbd0560a5e4f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:23.869138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46625d-547e-4807-ad89-fbd0560a5e4f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:23.870714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 224/420] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:40:23.876502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997f3ce-660e-4d0d-9c75-ff613cf76aef None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:23.876988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997f3ce-660e-4d0d-9c75-ff613cf76aef None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:23.877434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997f3ce-660e-4d0d-9c75-ff613cf76aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:23.902810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4997f3ce-660e-4d0d-9c75-ff613cf76aef None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:23.904525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4997f3ce-660e-4d0d-9c75-ff613cf76aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:23.982295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ab40d-b1c7-4338-885b-768debd1b74a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:24.049760 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 225/421] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:24.050518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d25ca-e9ef-4d90-a89b-5a0a7fa087f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:24.051756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d25ca-e9ef-4d90-a89b-5a0a7fa087f1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:24.052215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d25ca-e9ef-4d90-a89b-5a0a7fa087f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:24.052634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d25ca-e9ef-4d90-a89b-5a0a7fa087f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:24.140648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d25ca-e9ef-4d90-a89b-5a0a7fa087f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:24.142452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d25ca-e9ef-4d90-a89b-5a0a7fa087f1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:24.180120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:24.311145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 197/422] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:24.316459 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997f3ce-660e-4d0d-9c75-ff613cf76aef None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:24.332517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:24.406630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d25ca-e9ef-4d90-a89b-5a0a7fa087f1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:24.478393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997f3ce-660e-4d0d-9c75-ff613cf76aef None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:24.525642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488770b3-a176-42fe-877a-b0b3142cae9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:24.526277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488770b3-a176-42fe-877a-b0b3142cae9d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:24.526618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488770b3-a176-42fe-877a-b0b3142cae9d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:24.527393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488770b3-a176-42fe-877a-b0b3142cae9d None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:24.528616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488770b3-a176-42fe-877a-b0b3142cae9d None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:24.530141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488770b3-a176-42fe-877a-b0b3142cae9d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:24.532084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 226/423] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:24 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:24.537845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb56803-9b11-45e7-bcca-5c97b6b005a5 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:24.538295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb56803-9b11-45e7-bcca-5c97b6b005a5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:24.538444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb56803-9b11-45e7-bcca-5c97b6b005a5 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:24.543922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 227/424] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:24.543922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb56803-9b11-45e7-bcca-5c97b6b005a5 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:24.579524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d25ca-e9ef-4d90-a89b-5a0a7fa087f1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:24.609809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:24.622095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa700526-e7be-4da3-b1bc-a9ea1ac43544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:24.630440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa700526-e7be-4da3-b1bc-a9ea1ac43544 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:24.630440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa700526-e7be-4da3-b1bc-a9ea1ac43544 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:24.630440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa700526-e7be-4da3-b1bc-a9ea1ac43544 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:24.636369 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa700526-e7be-4da3-b1bc-a9ea1ac43544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-aa700526-e7be-4da3-b1bc-a9ea1ac43544 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:24.641491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:24.642522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:24.643518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 198/425] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:40:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:24.650620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfa651-d2ee-47fe-88d8-aeef914b98cd None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:24.651374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfa651-d2ee-47fe-88d8-aeef914b98cd None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:24.651374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfa651-d2ee-47fe-88d8-aeef914b98cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:24.654249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:24.671908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-bfcfa651-d2ee-47fe-88d8-aeef914b98cd None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:24.673556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-bfcfa651-d2ee-47fe-88d8-aeef914b98cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:24.690267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb56803-9b11-45e7-bcca-5c97b6b005a5 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:24.739144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba6152-f447-4eec-b745-f7053d99b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:24.740302 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba6152-f447-4eec-b745-f7053d99b178 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:24.740795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba6152-f447-4eec-b745-f7053d99b178 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:24.741672 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba6152-f447-4eec-b745-f7053d99b178 None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:24.743449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba6152-f447-4eec-b745-f7053d99b178 None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:24.745905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba6152-f447-4eec-b745-f7053d99b178 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:24.748826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 228/426] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:24 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:24.749915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba6152-f447-4eec-b745-f7053d99b178 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:24.761656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 199/427] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:24.761656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfa651-d2ee-47fe-88d8-aeef914b98cd None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:24.956310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba6152-f447-4eec-b745-f7053d99b178 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:24.960394 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfa651-d2ee-47fe-88d8-aeef914b98cd None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:24.983635 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:25.014408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d0344-944d-4a39-851b-6824f99e0e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:25.015275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0344-944d-4a39-851b-6824f99e0e31 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:25.015555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0344-944d-4a39-851b-6824f99e0e31 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:25.015829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0344-944d-4a39-851b-6824f99e0e31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:25.016593 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d0344-944d-4a39-851b-6824f99e0e31 None admin] RBAC: Adding query filter params (group.id=5a82c020be1a496484fae4ad4ba72814, role.id=3f0c34a1f93e49adb1757f315333cc48, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:25.016904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d0344-944d-4a39-851b-6824f99e0e31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:25.018345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d0344-944d-4a39-851b-6824f99e0e31 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:25.032319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 200/428] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 11:40:24 2022] GET /identity/v3/role_assignments?group.id=5a82c020be1a496484fae4ad4ba72814&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 611 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:25.043376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d1c40-8c94-4860-a344-ed10df8dc7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:25.044276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d1c40-8c94-4860-a344-ed10df8dc7cc None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:25.044701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d1c40-8c94-4860-a344-ed10df8dc7cc None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:25.045115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d1c40-8c94-4860-a344-ed10df8dc7cc None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:25.045936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d1c40-8c94-4860-a344-ed10df8dc7cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:25.046409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d1c40-8c94-4860-a344-ed10df8dc7cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:25.047923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d1c40-8c94-4860-a344-ed10df8dc7cc None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:25.060761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 229/429] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:25.066779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d0344-944d-4a39-851b-6824f99e0e31 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:25.082985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74fb82-7b1a-4ea3-9d1b-351fcf4c4443 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:25.083268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74fb82-7b1a-4ea3-9d1b-351fcf4c4443 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:25.083542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74fb82-7b1a-4ea3-9d1b-351fcf4c4443 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:25.085057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 230/430] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:25.101896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd340b-ee11-408d-b255-f600236ba32a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:25.102173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd340b-ee11-408d-b255-f600236ba32a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:25.102444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd340b-ee11-408d-b255-f600236ba32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:25.133558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:25.143540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1dbd340b-ee11-408d-b255-f600236ba32a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:25.145695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1dbd340b-ee11-408d-b255-f600236ba32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:25.207707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d0344-944d-4a39-851b-6824f99e0e31 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:25.239061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 231/431] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:25.240507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd340b-ee11-408d-b255-f600236ba32a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:25.265982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42e1ae-41ea-4984-91c3-77ad4b329dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:25.266610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42e1ae-41ea-4984-91c3-77ad4b329dcf None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:25.266896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42e1ae-41ea-4984-91c3-77ad4b329dcf None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:25.267166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42e1ae-41ea-4984-91c3-77ad4b329dcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:25.326071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:25.350808 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42e1ae-41ea-4984-91c3-77ad4b329dcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:25.352760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42e1ae-41ea-4984-91c3-77ad4b329dcf None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:25.401182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd340b-ee11-408d-b255-f600236ba32a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:25.453738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7af4ae-202b-4583-a901-426416d40a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:25.454749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7af4ae-202b-4583-a901-426416d40a9a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:25.455231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7af4ae-202b-4583-a901-426416d40a9a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:25.455698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7af4ae-202b-4583-a901-426416d40a9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:25.459398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:25.471365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7af4ae-202b-4583-a901-426416d40a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-5e7af4ae-202b-4583-a901-426416d40a9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:25.476859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:25.477802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 11:40:25.478611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:25.479219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 201/432] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:25.484279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 232/433] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:25.486142 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42e1ae-41ea-4984-91c3-77ad4b329dcf None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:25.493036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7af4ae-202b-4583-a901-426416d40a9a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:25.583818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:25.614411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42e1ae-41ea-4984-91c3-77ad4b329dcf None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:25.640953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7af4ae-202b-4583-a901-426416d40a9a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:25.658563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d6e72-fec8-41ff-a676-d2a6ebcbb219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:25.659209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d6e72-fec8-41ff-a676-d2a6ebcbb219 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:25.659490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d6e72-fec8-41ff-a676-d2a6ebcbb219 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:25.659769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d6e72-fec8-41ff-a676-d2a6ebcbb219 None admin] PATH_INFO: `/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:25.661216 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d6e72-fec8-41ff-a676-d2a6ebcbb219 None admin] RBAC: Authorizing `identity:get_project(project_id=973ebf7f6f414136a6ae5a7d1a1a919c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:25.662702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d6e72-fec8-41ff-a676-d2a6ebcbb219 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:25.664719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 202/434] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:25 2022] GET /identity/v3/projects/973ebf7f6f414136a6ae5a7d1a1a919c => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:25.665521 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d6e72-fec8-41ff-a676-d2a6ebcbb219 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:25.692579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8acf8d-4e29-4368-b97d-4b20c4f871ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:25.693194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8acf8d-4e29-4368-b97d-4b20c4f871ac None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:25.693477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8acf8d-4e29-4368-b97d-4b20c4f871ac None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:25.693751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8acf8d-4e29-4368-b97d-4b20c4f871ac None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:25.694435 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8acf8d-4e29-4368-b97d-4b20c4f871ac None admin] RBAC: Adding query filter params (name=member) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:25.694742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8acf8d-4e29-4368-b97d-4b20c4f871ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:25.696143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8acf8d-4e29-4368-b97d-4b20c4f871ac None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:25.706542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 233/435] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:40:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:25.707200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8acf8d-4e29-4368-b97d-4b20c4f871ac None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:25.800357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:25.804934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d6e72-fec8-41ff-a676-d2a6ebcbb219 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:25.847597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b134dd1-ec64-4721-868b-dbdc729866cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:25.848277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b134dd1-ec64-4721-868b-dbdc729866cf None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:25.848557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b134dd1-ec64-4721-868b-dbdc729866cf None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:25.848826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b134dd1-ec64-4721-868b-dbdc729866cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:25.856016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8acf8d-4e29-4368-b97d-4b20c4f871ac None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:25.862488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b134dd1-ec64-4721-868b-dbdc729866cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-9b134dd1-ec64-4721-868b-dbdc729866cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:25.867541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:25.868720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 203/436] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:40:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:25.869871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b134dd1-ec64-4721-868b-dbdc729866cf None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:25.880526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:25.898610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c98991-e1d4-412c-ab92-9bff29671b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:25.899254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98991-e1d4-412c-ab92-9bff29671b57 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:25.899540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98991-e1d4-412c-ab92-9bff29671b57 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:25.899822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98991-e1d4-412c-ab92-9bff29671b57 None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:25.901005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c98991-e1d4-412c-ab92-9bff29671b57 None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:25.902481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c98991-e1d4-412c-ab92-9bff29671b57 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:25.904217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 234/437] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:25 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:25.905072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c98991-e1d4-412c-ab92-9bff29671b57 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:25.985362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b134dd1-ec64-4721-868b-dbdc729866cf None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:26.009809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c98991-e1d4-412c-ab92-9bff29671b57 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:26.019065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de1e67-95eb-4005-abec-3ea633ff4326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:26.019671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de1e67-95eb-4005-abec-3ea633ff4326 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.019945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de1e67-95eb-4005-abec-3ea633ff4326 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.020242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de1e67-95eb-4005-abec-3ea633ff4326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.036948 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:26.046012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f54f4-b083-41b4-a156-8a83d2302a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:26.048855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f54f4-b083-41b4-a156-8a83d2302a23 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.048855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f54f4-b083-41b4-a156-8a83d2302a23 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.048855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f54f4-b083-41b4-a156-8a83d2302a23 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.048855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f54f4-b083-41b4-a156-8a83d2302a23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:26.048855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f54f4-b083-41b4-a156-8a83d2302a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:26.049591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f54f4-b083-41b4-a156-8a83d2302a23 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:26.058531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 235/438] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:26.059408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f54f4-b083-41b4-a156-8a83d2302a23 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:26.092620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de1e67-95eb-4005-abec-3ea633ff4326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:26.094092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de1e67-95eb-4005-abec-3ea633ff4326 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:26.102321 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:26.160590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f54f4-b083-41b4-a156-8a83d2302a23 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:26.194691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 204/439] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:26.194691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584175d8-4f64-400f-9b37-345f57374925 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.194691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584175d8-4f64-400f-9b37-345f57374925 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.194691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584175d8-4f64-400f-9b37-345f57374925 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.194691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 205/440] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:26.194691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584175d8-4f64-400f-9b37-345f57374925 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:26.201495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f2f72-e09b-4d4e-be68-3c7aaac05032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:26.202202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f2f72-e09b-4d4e-be68-3c7aaac05032 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.202553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f2f72-e09b-4d4e-be68-3c7aaac05032 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.202840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f2f72-e09b-4d4e-be68-3c7aaac05032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.203575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f2f72-e09b-4d4e-be68-3c7aaac05032 None admin] RBAC: Adding query filter params (group.id=5a82c020be1a496484fae4ad4ba72814, role.id=4bb04c67143448539c5af8d8bdd621f9, scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:26.203883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f2f72-e09b-4d4e-be68-3c7aaac05032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:26.205354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f2f72-e09b-4d4e-be68-3c7aaac05032 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:26.211334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 236/441] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 11:40:26 2022] GET /identity/v3/role_assignments?group.id=5a82c020be1a496484fae4ad4ba72814&scope.project.id=973ebf7f6f414136a6ae5a7d1a1a919c&role.id=4bb04c67143448539c5af8d8bdd621f9 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:26.212122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f2f72-e09b-4d4e-be68-3c7aaac05032 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:26.272922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:26.317074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584175d8-4f64-400f-9b37-345f57374925 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:26.325910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:26.334502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f2f72-e09b-4d4e-be68-3c7aaac05032 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:26.361003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f974a-c967-4d95-9e4d-a830baf40f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:26.362045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f974a-c967-4d95-9e4d-a830baf40f53 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.362718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f974a-c967-4d95-9e4d-a830baf40f53 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.362996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f974a-c967-4d95-9e4d-a830baf40f53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.374911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3f29e-0aed-4858-a5a9-c5baa9099ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:26.375557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3f29e-0aed-4858-a5a9-c5baa9099ec6 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.376144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3f29e-0aed-4858-a5a9-c5baa9099ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.376144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3f29e-0aed-4858-a5a9-c5baa9099ec6 None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.377316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3f29e-0aed-4858-a5a9-c5baa9099ec6 None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:26.378779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3f29e-0aed-4858-a5a9-c5baa9099ec6 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:26.385261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 237/442] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:26 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:26.385261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7f0e4-72f1-4ea9-a36c-cb1a273500a6 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.385261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7f0e4-72f1-4ea9-a36c-cb1a273500a6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.385261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7f0e4-72f1-4ea9-a36c-cb1a273500a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.403269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-6ea7f0e4-72f1-4ea9-a36c-cb1a273500a6 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:26.404744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6ea7f0e4-72f1-4ea9-a36c-cb1a273500a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:26.449775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f974a-c967-4d95-9e4d-a830baf40f53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:26.450396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f974a-c967-4d95-9e4d-a830baf40f53 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:26.496197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 238/443] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:26.496197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7f0e4-72f1-4ea9-a36c-cb1a273500a6 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:26.578495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:26.579332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 206/444] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:26.584830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f974a-c967-4d95-9e4d-a830baf40f53 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:26.595976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:26.650161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7f0e4-72f1-4ea9-a36c-cb1a273500a6 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:26.696103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22353d37-786b-43df-bdca-8465950635ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:26.696602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22353d37-786b-43df-bdca-8465950635ea None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.696602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22353d37-786b-43df-bdca-8465950635ea None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.696784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22353d37-786b-43df-bdca-8465950635ea None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.701106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22353d37-786b-43df-bdca-8465950635ea None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:26.702625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22353d37-786b-43df-bdca-8465950635ea None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:26.705029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 239/445] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:26 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:26.710989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22353d37-786b-43df-bdca-8465950635ea None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:26.743248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f974a-c967-4d95-9e4d-a830baf40f53 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:26.780608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3072a-88cf-4e29-ba7d-094427e1d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:26.781236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3072a-88cf-4e29-ba7d-094427e1d39d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.781681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3072a-88cf-4e29-ba7d-094427e1d39d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.781967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3072a-88cf-4e29-ba7d-094427e1d39d None admin] PATH_INFO: `/v3/projects/98f2008c4252437ea79b895753cd0c32` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.788019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3072a-88cf-4e29-ba7d-094427e1d39d None admin] RBAC: Authorizing `identity:get_project(project_id=98f2008c4252437ea79b895753cd0c32)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:26.789527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3072a-88cf-4e29-ba7d-094427e1d39d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:26.791207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 207/446] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:26 2022] GET /identity/v3/projects/98f2008c4252437ea79b895753cd0c32 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:26.791982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3072a-88cf-4e29-ba7d-094427e1d39d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:26.822833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22353d37-786b-43df-bdca-8465950635ea None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:26.841285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:26.858437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a2e70-3669-463e-90b6-d26051fa578e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:26.862279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a2e70-3669-463e-90b6-d26051fa578e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.862279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a2e70-3669-463e-90b6-d26051fa578e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.862279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a2e70-3669-463e-90b6-d26051fa578e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.862279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a2e70-3669-463e-90b6-d26051fa578e None admin] RBAC: Adding query filter params (group.id=5a82c020be1a496484fae4ad4ba72814, role.id=3f0c34a1f93e49adb1757f315333cc48, scope.project.id=98f2008c4252437ea79b895753cd0c32) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:26.862279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a2e70-3669-463e-90b6-d26051fa578e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:26.862279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a2e70-3669-463e-90b6-d26051fa578e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:26.867880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 240/447] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 11:40:26 2022] GET /identity/v3/role_assignments?group.id=5a82c020be1a496484fae4ad4ba72814&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=3f0c34a1f93e49adb1757f315333cc48 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:26.868716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a2e70-3669-463e-90b6-d26051fa578e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:26.873287 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:26.901672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3072a-88cf-4e29-ba7d-094427e1d39d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:26.941755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb78585-6806-41ee-ba12-3478989e047e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:26.942528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb78585-6806-41ee-ba12-3478989e047e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:26.942807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb78585-6806-41ee-ba12-3478989e047e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:26.943200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb78585-6806-41ee-ba12-3478989e047e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:26.957930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb78585-6806-41ee-ba12-3478989e047e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-0eb78585-6806-41ee-ba12-3478989e047e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:26.965834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:26.967070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:40:26.967991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:26.970077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 208/448] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:40:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:26.976342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb78585-6806-41ee-ba12-3478989e047e None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:27.010586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a2e70-3669-463e-90b6-d26051fa578e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:27.048722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1d362-04b1-4e31-8837-ae77571afd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:27.049365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1d362-04b1-4e31-8837-ae77571afd31 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:27.049958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1d362-04b1-4e31-8837-ae77571afd31 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:27.049958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1d362-04b1-4e31-8837-ae77571afd31 None admin] PATH_INFO: `/v3/groups/5a82c020be1a496484fae4ad4ba72814` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:27.051034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1d362-04b1-4e31-8837-ae77571afd31 None admin] RBAC: Authorizing `identity:get_group(group_id=5a82c020be1a496484fae4ad4ba72814)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:27.052553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1d362-04b1-4e31-8837-ae77571afd31 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:27.054109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 241/449] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:26 2022] GET /identity/v3/groups/5a82c020be1a496484fae4ad4ba72814 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:27.060803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1d362-04b1-4e31-8837-ae77571afd31 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:27.103026 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb78585-6806-41ee-ba12-3478989e047e None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:27.128025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:27.143990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c4878-1039-47a8-adf4-62be176093b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:27.145053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c4878-1039-47a8-adf4-62be176093b9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:27.145396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c4878-1039-47a8-adf4-62be176093b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:27.145818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c4878-1039-47a8-adf4-62be176093b9 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:27.147040 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c4878-1039-47a8-adf4-62be176093b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:27.147594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c4878-1039-47a8-adf4-62be176093b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:27.150143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c4878-1039-47a8-adf4-62be176093b9 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:27.152690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:27.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 209/450] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:27.168252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c4878-1039-47a8-adf4-62be176093b9 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:27.189719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1d362-04b1-4e31-8837-ae77571afd31 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:27.241467 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8860ca2-705c-43da-b240-409ab08a8f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:27.241467 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8860ca2-705c-43da-b240-409ab08a8f4b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:27.241842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8860ca2-705c-43da-b240-409ab08a8f4b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:27.242369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8860ca2-705c-43da-b240-409ab08a8f4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:27.242779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8860ca2-705c-43da-b240-409ab08a8f4b None admin] RBAC: Adding query filter params (group.id=5a82c020be1a496484fae4ad4ba72814, role.id=4bb04c67143448539c5af8d8bdd621f9, scope.project.id=98f2008c4252437ea79b895753cd0c32) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:27.243087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8860ca2-705c-43da-b240-409ab08a8f4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:27.245045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8860ca2-705c-43da-b240-409ab08a8f4b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:27.252382 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 242/451] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 11:40:27 2022] GET /identity/v3/role_assignments?group.id=5a82c020be1a496484fae4ad4ba72814&scope.project.id=98f2008c4252437ea79b895753cd0c32&role.id=4bb04c67143448539c5af8d8bdd621f9 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:27.298406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c4878-1039-47a8-adf4-62be176093b9 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:27.341386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18865940-03f7-46e0-ba73-4a0c38b0b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:27.343128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18865940-03f7-46e0-ba73-4a0c38b0b95b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:27.343128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18865940-03f7-46e0-ba73-4a0c38b0b95b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:27.343128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18865940-03f7-46e0-ba73-4a0c38b0b95b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:27.411183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:27.418756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18865940-03f7-46e0-ba73-4a0c38b0b95b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:27.420297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18865940-03f7-46e0-ba73-4a0c38b0b95b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:27.533811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 210/452] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 11:40:27.539751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8860ca2-705c-43da-b240-409ab08a8f4b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:27.647576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8860ca2-705c-43da-b240-409ab08a8f4b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:27.661735 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:27.681118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40e9ec-b347-4650-913a-1a82b1f1dd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:27.681733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40e9ec-b347-4650-913a-1a82b1f1dd2e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:27.682009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40e9ec-b347-4650-913a-1a82b1f1dd2e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:27.682276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40e9ec-b347-4650-913a-1a82b1f1dd2e None admin] PATH_INFO: `/v3/projects/a155f451dd324796b4bda64d212b8b97` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:27.683490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40e9ec-b347-4650-913a-1a82b1f1dd2e None admin] RBAC: Authorizing `identity:get_project(project_id=a155f451dd324796b4bda64d212b8b97)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:27.685102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40e9ec-b347-4650-913a-1a82b1f1dd2e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:27.690425 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 243/453] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:40:27 2022] GET /identity/v3/projects/a155f451dd324796b4bda64d212b8b97 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:27.692426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18865940-03f7-46e0-ba73-4a0c38b0b95b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:27.785992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:27.793163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18865940-03f7-46e0-ba73-4a0c38b0b95b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:27.827450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d34122-af55-4cfc-af7f-91c9087fadb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:27.828072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d34122-af55-4cfc-af7f-91c9087fadb8 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:27.828371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d34122-af55-4cfc-af7f-91c9087fadb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:27.828644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d34122-af55-4cfc-af7f-91c9087fadb8 None admin] PATH_INFO: `/v3/groups/18499fcfc0314aa5882522f9e8069100` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:27.829778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d34122-af55-4cfc-af7f-91c9087fadb8 None admin] RBAC: Authorizing `identity:get_group(group_id=18499fcfc0314aa5882522f9e8069100)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:27.831200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d34122-af55-4cfc-af7f-91c9087fadb8 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:27.832786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 211/454] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Oct 4 11:40:27 2022] GET /identity/v3/groups/18499fcfc0314aa5882522f9e8069100 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:27.840473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40e9ec-b347-4650-913a-1a82b1f1dd2e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:27.940927 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40e9ec-b347-4650-913a-1a82b1f1dd2e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:27.974633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea56cd-5d76-4be8-96ec-d30ce9f00431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:27.975235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea56cd-5d76-4be8-96ec-d30ce9f00431 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:27.975517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea56cd-5d76-4be8-96ec-d30ce9f00431 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:27.975794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea56cd-5d76-4be8-96ec-d30ce9f00431 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:27.976566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea56cd-5d76-4be8-96ec-d30ce9f00431 None admin] RBAC: Adding query filter params (group.id=18499fcfc0314aa5882522f9e8069100, role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=a155f451dd324796b4bda64d212b8b97) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:27.976878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea56cd-5d76-4be8-96ec-d30ce9f00431 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:27.978327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea56cd-5d76-4be8-96ec-d30ce9f00431 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:27.984736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 244/455] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Oct 4 11:40:27 2022] GET /identity/v3/role_assignments?group.id=18499fcfc0314aa5882522f9e8069100&scope.project.id=a155f451dd324796b4bda64d212b8b97&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:30.878350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cffd4-c779-4522-87ed-da042f346482 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:30.878939 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cffd4-c779-4522-87ed-da042f346482 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:30.878939 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cffd4-c779-4522-87ed-da042f346482 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:30.880585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 212/456] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:30.886096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf9b0f-48b0-4d48-ab4a-3a617747e542 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:30.887055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf9b0f-48b0-4d48-ab4a-3a617747e542 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:30.887568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf9b0f-48b0-4d48-ab4a-3a617747e542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:30.897328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f95fb-64fa-4145-85f7-f619a2b6282c None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:30.897641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f95fb-64fa-4145-85f7-f619a2b6282c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:30.897870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f95fb-64fa-4145-85f7-f619a2b6282c None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:30.899637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 213/457] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:30.911620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abd97f-9cff-4bf1-88d3-fdc0c4f60d0f None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:30.912356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abd97f-9cff-4bf1-88d3-fdc0c4f60d0f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:30.912870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abd97f-9cff-4bf1-88d3-fdc0c4f60d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:30.915643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2ddf9b0f-48b0-4d48-ab4a-3a617747e542 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:30.917973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2ddf9b0f-48b0-4d48-ab4a-3a617747e542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:30.942474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b5abd97f-9cff-4bf1-88d3-fdc0c4f60d0f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:30.943734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b5abd97f-9cff-4bf1-88d3-fdc0c4f60d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:30.977342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:31.019666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 245/458] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:31.054956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef304950-67bb-400a-9d77-271a22ad4e59 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:31.055870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef304950-67bb-400a-9d77-271a22ad4e59 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:31.057348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef304950-67bb-400a-9d77-271a22ad4e59 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:31.062263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 246/459] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:31.070994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaf019-bd2d-4aca-b710-e141cd309159 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:31.071400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaf019-bd2d-4aca-b710-e141cd309159 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:31.071573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaf019-bd2d-4aca-b710-e141cd309159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:31.091773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 214/460] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:31.104907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-04eaf019-bd2d-4aca-b710-e141cd309159 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:31.106129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-04eaf019-bd2d-4aca-b710-e141cd309159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:31.208577 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 247/461] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:31.346838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abd97f-9cff-4bf1-88d3-fdc0c4f60d0f None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:31.373788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaf019-bd2d-4aca-b710-e141cd309159 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:31.521105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:31.534709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abd97f-9cff-4bf1-88d3-fdc0c4f60d0f None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:31.542567 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaf019-bd2d-4aca-b710-e141cd309159 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:31.557730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:31.599070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9971bb4-ec6b-412f-b1ab-2ed03f708579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:31.600129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9971bb4-ec6b-412f-b1ab-2ed03f708579 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:31.600913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9971bb4-ec6b-412f-b1ab-2ed03f708579 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:31.601347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9971bb4-ec6b-412f-b1ab-2ed03f708579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:31.611900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03e6c7-7f6b-4d45-8196-c86fde4fc8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:31.612947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03e6c7-7f6b-4d45-8196-c86fde4fc8bf None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:31.613432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03e6c7-7f6b-4d45-8196-c86fde4fc8bf None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:31.613866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03e6c7-7f6b-4d45-8196-c86fde4fc8bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:31.614437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9971bb4-ec6b-412f-b1ab-2ed03f708579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:31.616510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9971bb4-ec6b-412f-b1ab-2ed03f708579 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:31.628713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03e6c7-7f6b-4d45-8196-c86fde4fc8bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:31.630698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03e6c7-7f6b-4d45-8196-c86fde4fc8bf None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-f9971bb4-ec6b-412f-b1ab-2ed03f708579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:31.636304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:31.637537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:31.638481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:31.638481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:31.638481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:31.638679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 248/462] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:31.639689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9971bb4-ec6b-412f-b1ab-2ed03f708579 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-da03e6c7-7f6b-4d45-8196-c86fde4fc8bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:31.655876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:31.657096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:31.658085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:31.658085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:31.658085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:31.658085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 215/463] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:31.658867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03e6c7-7f6b-4d45-8196-c86fde4fc8bf None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:31.775097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9971bb4-ec6b-412f-b1ab-2ed03f708579 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:31.775329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03e6c7-7f6b-4d45-8196-c86fde4fc8bf None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:31.817594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1d8de-dfca-482c-98e6-6c96da90376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:31.818191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1d8de-dfca-482c-98e6-6c96da90376c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:31.818467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1d8de-dfca-482c-98e6-6c96da90376c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:31.818736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1d8de-dfca-482c-98e6-6c96da90376c None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:31.819346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47999c-d49b-4001-be2f-0a2ecd0ecb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:31.819487 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1d8de-dfca-482c-98e6-6c96da90376c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:31.819659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1d8de-dfca-482c-98e6-6c96da90376c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:31.819979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47999c-d49b-4001-be2f-0a2ecd0ecb21 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:31.820299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47999c-d49b-4001-be2f-0a2ecd0ecb21 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:31.820578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47999c-d49b-4001-be2f-0a2ecd0ecb21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:31.821151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1d8de-dfca-482c-98e6-6c96da90376c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:31.832875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47999c-d49b-4001-be2f-0a2ecd0ecb21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:31.834159 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47999c-d49b-4001-be2f-0a2ecd0ecb21 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:31.838872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 216/464] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:31.840199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1d8de-dfca-482c-98e6-6c96da90376c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:31.852467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-2d47999c-d49b-4001-be2f-0a2ecd0ecb21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:31.853082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:31.854187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:31.855800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:31.855800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:31.855800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:31.860628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 249/465] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:31.862111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949558c-8331-4f6b-89c6-3dd85370c1fb None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:31.862386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949558c-8331-4f6b-89c6-3dd85370c1fb None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:31.862704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949558c-8331-4f6b-89c6-3dd85370c1fb None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:31.864071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 250/466] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:31.869041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949558c-8331-4f6b-89c6-3dd85370c1fb None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:31.883959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:31.973015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1d8de-dfca-482c-98e6-6c96da90376c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:31.992493 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949558c-8331-4f6b-89c6-3dd85370c1fb None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:32.014613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac8c26-791a-4286-9014-c920789719ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:32.015236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac8c26-791a-4286-9014-c920789719ea None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:32.015519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac8c26-791a-4286-9014-c920789719ea None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:32.015795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac8c26-791a-4286-9014-c920789719ea None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:32.016495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac8c26-791a-4286-9014-c920789719ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:32.016809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac8c26-791a-4286-9014-c920789719ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:32.018244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac8c26-791a-4286-9014-c920789719ea None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:32.032624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 217/467] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:32.033395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac8c26-791a-4286-9014-c920789719ea None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:32.039086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2846823-4893-433e-8e45-8b32f86c3e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:32.039719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2846823-4893-433e-8e45-8b32f86c3e3a None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:32.040002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2846823-4893-433e-8e45-8b32f86c3e3a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:32.040333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2846823-4893-433e-8e45-8b32f86c3e3a None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:32.041012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2846823-4893-433e-8e45-8b32f86c3e3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:32.042996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2846823-4893-433e-8e45-8b32f86c3e3a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:32.114107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:32.150519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac8c26-791a-4286-9014-c920789719ea None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:32.157451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 251/468] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:40:31 2022] POST /identity/v3/users => generated 248 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:40:32.158716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7e036-c9c1-4f75-9da2-6ec1f6618596 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:32.159016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7e036-c9c1-4f75-9da2-6ec1f6618596 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:32.159293 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7e036-c9c1-4f75-9da2-6ec1f6618596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:32.196874 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d0b7e036-c9c1-4f75-9da2-6ec1f6618596 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:32.198869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d0b7e036-c9c1-4f75-9da2-6ec1f6618596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:32.210960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03084758-886e-405d-ad68-ad8ac2e64354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:32.211696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03084758-886e-405d-ad68-ad8ac2e64354 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:32.211978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03084758-886e-405d-ad68-ad8ac2e64354 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:32.212674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03084758-886e-405d-ad68-ad8ac2e64354 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:32.217726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03084758-886e-405d-ad68-ad8ac2e64354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:32.217726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03084758-886e-405d-ad68-ad8ac2e64354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:32.217726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03084758-886e-405d-ad68-ad8ac2e64354 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:32.234373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 218/469] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:32.235537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03084758-886e-405d-ad68-ad8ac2e64354 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:32.286302 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:32.295598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 252/470] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:32.296559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7e036-c9c1-4f75-9da2-6ec1f6618596 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:32.377509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03084758-886e-405d-ad68-ad8ac2e64354 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:32.435852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:32.438370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7e036-c9c1-4f75-9da2-6ec1f6618596 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:32.455462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac04310-bc6c-4c15-8d4f-1be8f83696a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:32.456087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac04310-bc6c-4c15-8d4f-1be8f83696a6 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:32.456417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac04310-bc6c-4c15-8d4f-1be8f83696a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:32.457267 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac04310-bc6c-4c15-8d4f-1be8f83696a6 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:32.458014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac04310-bc6c-4c15-8d4f-1be8f83696a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:32.459964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac04310-bc6c-4c15-8d4f-1be8f83696a6 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:32.503804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbe360-81d3-4ea5-a064-fcf425afaeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:32.504882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbe360-81d3-4ea5-a064-fcf425afaeb9 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:32.505567 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbe360-81d3-4ea5-a064-fcf425afaeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:32.505831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbe360-81d3-4ea5-a064-fcf425afaeb9 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:32.506988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbe360-81d3-4ea5-a064-fcf425afaeb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:32.509684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbe360-81d3-4ea5-a064-fcf425afaeb9 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:32.531063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 219/471] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:40:32 2022] POST /identity/v3/users => generated 246 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:40:32.534044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daae26-5dda-46a4-b0fb-e1320eebd5a2 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:32.534044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daae26-5dda-46a4-b0fb-e1320eebd5a2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:32.534044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daae26-5dda-46a4-b0fb-e1320eebd5a2 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:32.534948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 220/472] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:32.541261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0fae1-ea9e-4bf3-9e0d-16792c2fba1d None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:32.541561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0fae1-ea9e-4bf3-9e0d-16792c2fba1d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:32.542415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0fae1-ea9e-4bf3-9e0d-16792c2fba1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:32.566130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d3e0fae1-ea9e-4bf3-9e0d-16792c2fba1d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:32.566629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d3e0fae1-ea9e-4bf3-9e0d-16792c2fba1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:32.597416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 253/473] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:40:32 2022] POST /identity/v3/users => generated 248 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:40:32.642383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbe360-81d3-4ea5-a064-fcf425afaeb9 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:32.684857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 221/474] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:32.742062 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:32.824582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbe360-81d3-4ea5-a064-fcf425afaeb9 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:32.882377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde4d89-e8b2-4d14-a4ef-3d453764636a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:32.883031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde4d89-e8b2-4d14-a4ef-3d453764636a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:32.883316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde4d89-e8b2-4d14-a4ef-3d453764636a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:32.883595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde4d89-e8b2-4d14-a4ef-3d453764636a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:32.897881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde4d89-e8b2-4d14-a4ef-3d453764636a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:32.899647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde4d89-e8b2-4d14-a4ef-3d453764636a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-dbde4d89-e8b2-4d14-a4ef-3d453764636a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:32.917881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:32.919018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:32.920076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:32.920076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:32.920076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:32.921520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 254/475] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:32.928146 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0fae1-ea9e-4bf3-9e0d-16792c2fba1d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:33.058604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde4d89-e8b2-4d14-a4ef-3d453764636a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:33.060857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0fae1-ea9e-4bf3-9e0d-16792c2fba1d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:33.081523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:33.109185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f1f69-0c34-4ddc-a004-13ebc1d2e0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:33.110109 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f1f69-0c34-4ddc-a004-13ebc1d2e0d4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:33.110550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f1f69-0c34-4ddc-a004-13ebc1d2e0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:33.110969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f1f69-0c34-4ddc-a004-13ebc1d2e0d4 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:33.111776 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f1f69-0c34-4ddc-a004-13ebc1d2e0d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:33.112279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f1f69-0c34-4ddc-a004-13ebc1d2e0d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:33.113824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f1f69-0c34-4ddc-a004-13ebc1d2e0d4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:33.127958 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 222/476] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:33.133570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f1f69-0c34-4ddc-a004-13ebc1d2e0d4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:33.176921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde4d89-e8b2-4d14-a4ef-3d453764636a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:33.180668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:33.220951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33817d6-ad5c-4ea9-8c71-defa9a32933b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:33.221727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33817d6-ad5c-4ea9-8c71-defa9a32933b None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:33.222013 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33817d6-ad5c-4ea9-8c71-defa9a32933b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:33.222309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33817d6-ad5c-4ea9-8c71-defa9a32933b None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:33.223373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33817d6-ad5c-4ea9-8c71-defa9a32933b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:33.225333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33817d6-ad5c-4ea9-8c71-defa9a32933b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:33.270481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.sql.core [None req-b33817d6-ad5c-4ea9-8c71-defa9a32933b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 11:40:33.270481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 11:40:33.270481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [parameters: {'id': '8d836de0be5346bb992afa81e816d24c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 11:40:33.270481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93212) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-b33817d6-ad5c-4ea9-8c71-defa9a32933b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application conn.query(q) Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application result.read() Oct 04 11:40:33.307323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return ref.to_dict() Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 11:40:33.308409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application next(self.gen) Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application yield resource Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application next(self.gen) Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise self.value Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application session.commit() Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 11:40:33.309414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.session.flush() Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application compat.raise_( Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 11:40:33.310289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 11:40:33.311226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application conn.query(q) Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application result.read() Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 11:40:33.312098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [parameters: {'id': '8d836de0be5346bb992afa81e816d24c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:33.312954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 11:40:33.313755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:33.314628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 255/477] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 4 11:40:33 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:33.316139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33817d6-ad5c-4ea9-8c71-defa9a32933b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:33.421075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f1f69-0c34-4ddc-a004-13ebc1d2e0d4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:33.499991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e14190-f3ca-47b1-92e4-b1fb034b8e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:33.500551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e14190-f3ca-47b1-92e4-b1fb034b8e0b None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:33.500858 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e14190-f3ca-47b1-92e4-b1fb034b8e0b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:33.501131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e14190-f3ca-47b1-92e4-b1fb034b8e0b None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:33.501895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e14190-f3ca-47b1-92e4-b1fb034b8e0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:33.503337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e14190-f3ca-47b1-92e4-b1fb034b8e0b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:33.528637 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33817d6-ad5c-4ea9-8c71-defa9a32933b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:33.565651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9be35-9ab1-435a-83cf-985b5e1a31d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:33.566284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9be35-9ab1-435a-83cf-985b5e1a31d1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:33.566560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9be35-9ab1-435a-83cf-985b5e1a31d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:33.566829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9be35-9ab1-435a-83cf-985b5e1a31d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:33.578278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9be35-9ab1-435a-83cf-985b5e1a31d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-5fa9be35-9ab1-435a-83cf-985b5e1a31d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:33.584135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:33.585132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:33.586026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 256/479] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:40:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:33.588397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 223/479] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:40:33 2022] POST /identity/v3/users => generated 249 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:40:33.592757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9be35-9ab1-435a-83cf-985b5e1a31d1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:33.647465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:33.764726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9be35-9ab1-435a-83cf-985b5e1a31d1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:33.840998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bd0cd-d3a4-4138-b457-d64defef394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:33.841635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bd0cd-d3a4-4138-b457-d64defef394a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:33.841916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bd0cd-d3a4-4138-b457-d64defef394a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:33.842192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bd0cd-d3a4-4138-b457-d64defef394a None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:33.842888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bd0cd-d3a4-4138-b457-d64defef394a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:33.843195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bd0cd-d3a4-4138-b457-d64defef394a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:33.844639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bd0cd-d3a4-4138-b457-d64defef394a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:33.855084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 257/480] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:35.000980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f8281-0a69-4c61-9cb3-69e2739e7513 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:35.001983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f8281-0a69-4c61-9cb3-69e2739e7513 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:35.002483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f8281-0a69-4c61-9cb3-69e2739e7513 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:35.004438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 224/481] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:35.009690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b0267-83f7-4b04-bd2a-da3d1f06f06b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:35.011121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b0267-83f7-4b04-bd2a-da3d1f06f06b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:35.011592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b0267-83f7-4b04-bd2a-da3d1f06f06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:35.048949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-781b0267-83f7-4b04-bd2a-da3d1f06f06b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:35.050959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-781b0267-83f7-4b04-bd2a-da3d1f06f06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:35.200361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 258/482] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:35.557438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f8281-0a69-4c61-9cb3-69e2739e7513 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:35.594076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:35.692460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f8281-0a69-4c61-9cb3-69e2739e7513 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:35.740670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e119e9-1122-4682-af50-1d9cc61fbae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:35.741326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e119e9-1122-4682-af50-1d9cc61fbae9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:35.741603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e119e9-1122-4682-af50-1d9cc61fbae9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:35.741880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e119e9-1122-4682-af50-1d9cc61fbae9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:35.757223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e119e9-1122-4682-af50-1d9cc61fbae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-b4e119e9-1122-4682-af50-1d9cc61fbae9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:35.763274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:35.765257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:35.767018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 225/483] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:35.773418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b0267-83f7-4b04-bd2a-da3d1f06f06b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:35.820837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:35.907175 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c91b08-83b9-49d5-8b83-31bf3f93d7a8 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:35.907930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c91b08-83b9-49d5-8b83-31bf3f93d7a8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:35.908481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c91b08-83b9-49d5-8b83-31bf3f93d7a8 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:35.910954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 226/484] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:40:35.920879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c3aa0-db0a-4ece-ac69-7642263bb4c7 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:35.921500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c3aa0-db0a-4ece-ac69-7642263bb4c7 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:35.921981 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c3aa0-db0a-4ece-ac69-7642263bb4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:35.943614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-234c3aa0-db0a-4ece-ac69-7642263bb4c7 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:35.945334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-234c3aa0-db0a-4ece-ac69-7642263bb4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:35.955094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b0267-83f7-4b04-bd2a-da3d1f06f06b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:36.030264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 227/485] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:40:36.046835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9f1e3-bc1f-46a8-8705-8c52566ba61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:36.048761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9f1e3-bc1f-46a8-8705-8c52566ba61c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:36.049438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9f1e3-bc1f-46a8-8705-8c52566ba61c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:36.050099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9f1e3-bc1f-46a8-8705-8c52566ba61c None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:36.051465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9f1e3-bc1f-46a8-8705-8c52566ba61c None admin] RBAC: Adding query filter params (name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:36.052221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9f1e3-bc1f-46a8-8705-8c52566ba61c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:36.054885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9f1e3-bc1f-46a8-8705-8c52566ba61c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:36.075111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 259/486] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:36.081393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c3aa0-db0a-4ece-ac69-7642263bb4c7 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:36.118056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:36.247509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c3aa0-db0a-4ece-ac69-7642263bb4c7 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:36.305072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dcf3c-0608-4c47-b6b8-8fccbd02afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:36.305739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dcf3c-0608-4c47-b6b8-8fccbd02afdb None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:36.306025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dcf3c-0608-4c47-b6b8-8fccbd02afdb None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:36.306301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dcf3c-0608-4c47-b6b8-8fccbd02afdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:36.318643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91dcf3c-0608-4c47-b6b8-8fccbd02afdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:36.320124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91dcf3c-0608-4c47-b6b8-8fccbd02afdb None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-c91dcf3c-0608-4c47-b6b8-8fccbd02afdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:36.344675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:36.345626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:36.347079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:36.347079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:36.347079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:36.347079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 228/487] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:36.351550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9f1e3-bc1f-46a8-8705-8c52566ba61c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:36.388268 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dcf3c-0608-4c47-b6b8-8fccbd02afdb None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:36.458059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:36.478984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9f1e3-bc1f-46a8-8705-8c52566ba61c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:36.496548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:36.530792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42406134-30b3-40bc-8c90-59094e42ff48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:36.531603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42406134-30b3-40bc-8c90-59094e42ff48 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:36.531955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42406134-30b3-40bc-8c90-59094e42ff48 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:36.532392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42406134-30b3-40bc-8c90-59094e42ff48 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:36.533163 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42406134-30b3-40bc-8c90-59094e42ff48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:36.533570 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42406134-30b3-40bc-8c90-59094e42ff48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:36.535089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42406134-30b3-40bc-8c90-59094e42ff48 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:36.542521 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dcf3c-0608-4c47-b6b8-8fccbd02afdb None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:36.550179 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 260/488] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:36.551658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6c7cb-d05e-4b11-9c14-f69109ab3c7a None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:36.552056 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6c7cb-d05e-4b11-9c14-f69109ab3c7a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:36.552421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6c7cb-d05e-4b11-9c14-f69109ab3c7a None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:36.554033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 261/489] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:36.559738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79031f75-48cb-4757-9f9a-536d2fbeb600 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:36.560025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79031f75-48cb-4757-9f9a-536d2fbeb600 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:36.560331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79031f75-48cb-4757-9f9a-536d2fbeb600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:36.580420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-79031f75-48cb-4757-9f9a-536d2fbeb600 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:36.581639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-79031f75-48cb-4757-9f9a-536d2fbeb600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:36.595020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8adc3-2e37-463f-8d37-8007a73484ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:36.596233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8adc3-2e37-463f-8d37-8007a73484ce None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:36.596662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8adc3-2e37-463f-8d37-8007a73484ce None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:36.597415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8adc3-2e37-463f-8d37-8007a73484ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:36.614218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8adc3-2e37-463f-8d37-8007a73484ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-30f8adc3-2e37-463f-8d37-8007a73484ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:36.622165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:36.623270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:36.624309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:36.626788 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 229/490] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:36.632433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8adc3-2e37-463f-8d37-8007a73484ce None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:36.659468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 262/491] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:36.660734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603bdb-3a38-4e15-9b58-44a4200768b9 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:36.661025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603bdb-3a38-4e15-9b58-44a4200768b9 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:36.661303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603bdb-3a38-4e15-9b58-44a4200768b9 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:36.662653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 263/492] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:36.668042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7603bdb-3a38-4e15-9b58-44a4200768b9 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:36.755376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:36.773863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8adc3-2e37-463f-8d37-8007a73484ce None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:36.798266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7603bdb-3a38-4e15-9b58-44a4200768b9 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:36.816106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:36.821902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631b7d0-f003-4fb2-9870-e7844b027e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:36.822558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631b7d0-f003-4fb2-9870-e7844b027e05 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:36.823842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631b7d0-f003-4fb2-9870-e7844b027e05 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:36.828223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631b7d0-f003-4fb2-9870-e7844b027e05 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:36.836040 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631b7d0-f003-4fb2-9870-e7844b027e05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:36.838242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631b7d0-f003-4fb2-9870-e7844b027e05 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:36.847047 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb75860-dd6f-492d-bc18-565695976fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:36.847964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb75860-dd6f-492d-bc18-565695976fd1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:36.848484 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb75860-dd6f-492d-bc18-565695976fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:36.848975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb75860-dd6f-492d-bc18-565695976fd1 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:36.849893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb75860-dd6f-492d-bc18-565695976fd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:36.850448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb75860-dd6f-492d-bc18-565695976fd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:36.852210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb75860-dd6f-492d-bc18-565695976fd1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-3631b7d0-f003-4fb2-9870-e7844b027e05 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:36.852698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:36.854373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 230/493] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:36.856072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00709ac-a391-4c78-b24d-622fca243ae4 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:36.859347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00709ac-a391-4c78-b24d-622fca243ae4 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:36.859347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00709ac-a391-4c78-b24d-622fca243ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:36.876621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 264/494] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:36.878021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb75860-dd6f-492d-bc18-565695976fd1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:36.879090 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b00709ac-a391-4c78-b24d-622fca243ae4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:36.880071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b00709ac-a391-4c78-b24d-622fca243ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:36.969541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 231/495] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:36.970417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00709ac-a391-4c78-b24d-622fca243ae4 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:37.001911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb75860-dd6f-492d-bc18-565695976fd1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:37.038888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1264473-ba15-41f3-bb65-de5b30c84f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:37.039501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1264473-ba15-41f3-bb65-de5b30c84f44 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:37.039783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1264473-ba15-41f3-bb65-de5b30c84f44 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:37.040057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1264473-ba15-41f3-bb65-de5b30c84f44 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:37.040841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1264473-ba15-41f3-bb65-de5b30c84f44 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:37.041150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1264473-ba15-41f3-bb65-de5b30c84f44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:37.043256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1264473-ba15-41f3-bb65-de5b30c84f44 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:37.056570 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:37.088012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 265/496] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 11:40:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:37.089004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1264473-ba15-41f3-bb65-de5b30c84f44 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:37.103050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00709ac-a391-4c78-b24d-622fca243ae4 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:37.134471 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:37.153760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7cff0-e5d1-4e8b-944b-6effca33fdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:37.154741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7cff0-e5d1-4e8b-944b-6effca33fdbb None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:37.155317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7cff0-e5d1-4e8b-944b-6effca33fdbb None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:37.155806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7cff0-e5d1-4e8b-944b-6effca33fdbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:37.168288 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7cff0-e5d1-4e8b-944b-6effca33fdbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:37.170166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7cff0-e5d1-4e8b-944b-6effca33fdbb None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-b4d7cff0-e5d1-4e8b-944b-6effca33fdbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:37.187063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:37.188054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:37.188977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:37.188977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:37.188977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:37.191358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 232/497] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:37.192577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7cff0-e5d1-4e8b-944b-6effca33fdbb None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:37.203859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1264473-ba15-41f3-bb65-de5b30c84f44 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:37.244591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164d55d-311c-4772-b489-ffc640d32165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:37.245299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d55d-311c-4772-b489-ffc640d32165 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:37.245582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d55d-311c-4772-b489-ffc640d32165 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:37.245888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d55d-311c-4772-b489-ffc640d32165 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:37.257796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164d55d-311c-4772-b489-ffc640d32165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-9164d55d-311c-4772-b489-ffc640d32165 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:37.263306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:37.264824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 266/498] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:37.265977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164d55d-311c-4772-b489-ffc640d32165 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:37.314596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7cff0-e5d1-4e8b-944b-6effca33fdbb None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:37.323953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:37.352187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242497f2-27ec-49af-9743-c3042c20eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:37.352822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242497f2-27ec-49af-9743-c3042c20eec8 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:37.353159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242497f2-27ec-49af-9743-c3042c20eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:37.353384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242497f2-27ec-49af-9743-c3042c20eec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:37.375835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164d55d-311c-4772-b489-ffc640d32165 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:37.399348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:37.412355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa88312-89d8-4d93-b701-8d3213dd9b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:37.412991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa88312-89d8-4d93-b701-8d3213dd9b79 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:37.413278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa88312-89d8-4d93-b701-8d3213dd9b79 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:37.413549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa88312-89d8-4d93-b701-8d3213dd9b79 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:37.424984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa88312-89d8-4d93-b701-8d3213dd9b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:37.424984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa88312-89d8-4d93-b701-8d3213dd9b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:37.424984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa88312-89d8-4d93-b701-8d3213dd9b79 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:37.427549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242497f2-27ec-49af-9743-c3042c20eec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:37.429010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 267/499] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:37.429751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa88312-89d8-4d93-b701-8d3213dd9b79 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:37.433308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242497f2-27ec-49af-9743-c3042c20eec8 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:37.547684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 233/500] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:37.547684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242497f2-27ec-49af-9743-c3042c20eec8 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:37.552468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa88312-89d8-4d93-b701-8d3213dd9b79 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:37.567505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:37.589646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86cd40-a8a8-49a5-b671-b37f1ad6d007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:37.590686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86cd40-a8a8-49a5-b671-b37f1ad6d007 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:37.591206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86cd40-a8a8-49a5-b671-b37f1ad6d007 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:37.591627 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86cd40-a8a8-49a5-b671-b37f1ad6d007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:37.608545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86cd40-a8a8-49a5-b671-b37f1ad6d007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:37.610070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86cd40-a8a8-49a5-b671-b37f1ad6d007 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-ac86cd40-a8a8-49a5-b671-b37f1ad6d007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:37.626405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:37.627378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:37.628271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:37.628271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:37.628271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:37.628271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 268/501] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:37.631214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:37.633230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86cd40-a8a8-49a5-b671-b37f1ad6d007 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:37.668015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242497f2-27ec-49af-9743-c3042c20eec8 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:37.709925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3775e3-128a-43be-8535-dac49fff6a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:37.710539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3775e3-128a-43be-8535-dac49fff6a5a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:37.710835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3775e3-128a-43be-8535-dac49fff6a5a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:37.711088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3775e3-128a-43be-8535-dac49fff6a5a None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:37.711766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3775e3-128a-43be-8535-dac49fff6a5a None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:37.712067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3775e3-128a-43be-8535-dac49fff6a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:37.713532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3775e3-128a-43be-8535-dac49fff6a5a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:37.728139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 234/502] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:37.728139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3775e3-128a-43be-8535-dac49fff6a5a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:37.763768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86cd40-a8a8-49a5-b671-b37f1ad6d007 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:37.809696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d686057-27e1-4121-b7f1-161b059dd3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:37.810405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d686057-27e1-4121-b7f1-161b059dd3d3 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:37.810691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d686057-27e1-4121-b7f1-161b059dd3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:37.810968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d686057-27e1-4121-b7f1-161b059dd3d3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:37.818808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d686057-27e1-4121-b7f1-161b059dd3d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:37.820299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d686057-27e1-4121-b7f1-161b059dd3d3 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-8d686057-27e1-4121-b7f1-161b059dd3d3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:37.833763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:37.834730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:37.835625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 269/503] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:37.836201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d686057-27e1-4121-b7f1-161b059dd3d3 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:37.848284 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3775e3-128a-43be-8535-dac49fff6a5a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:37.866018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:37.884111 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09bd76-ed11-4468-af88-0ffc7c0bbc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:37.884771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09bd76-ed11-4468-af88-0ffc7c0bbc78 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:37.885051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09bd76-ed11-4468-af88-0ffc7c0bbc78 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:37.885883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09bd76-ed11-4468-af88-0ffc7c0bbc78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:37.887964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:37.897886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09bd76-ed11-4468-af88-0ffc7c0bbc78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:37.899455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09bd76-ed11-4468-af88-0ffc7c0bbc78 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-8a09bd76-ed11-4468-af88-0ffc7c0bbc78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:37.920579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:37.921750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:37.922794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:37.922794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:37.922794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:37.927495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 235/504] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:37.928501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09bd76-ed11-4468-af88-0ffc7c0bbc78 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:37.964142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d686057-27e1-4121-b7f1-161b059dd3d3 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:38.003591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35e534-7f58-4b7e-a54a-36f3b1eda0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:38.004384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e534-7f58-4b7e-a54a-36f3b1eda0b7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.004662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e534-7f58-4b7e-a54a-36f3b1eda0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.004929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e534-7f58-4b7e-a54a-36f3b1eda0b7 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.005574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35e534-7f58-4b7e-a54a-36f3b1eda0b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:38.005877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35e534-7f58-4b7e-a54a-36f3b1eda0b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:38.007628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35e534-7f58-4b7e-a54a-36f3b1eda0b7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:38.021030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 270/505] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:38.021735 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35e534-7f58-4b7e-a54a-36f3b1eda0b7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:38.044693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09bd76-ed11-4468-af88-0ffc7c0bbc78 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:38.080322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886197ee-4f68-4bf4-b77c-1b1abe8612f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:38.080956 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886197ee-4f68-4bf4-b77c-1b1abe8612f7 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.081235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886197ee-4f68-4bf4-b77c-1b1abe8612f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.081507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886197ee-4f68-4bf4-b77c-1b1abe8612f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.092955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886197ee-4f68-4bf4-b77c-1b1abe8612f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:38.094432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886197ee-4f68-4bf4-b77c-1b1abe8612f7 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-886197ee-4f68-4bf4-b77c-1b1abe8612f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:38.109911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:38.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:38.113211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:38.113211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:38.113211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:38.113211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 236/506] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:38.113211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886197ee-4f68-4bf4-b77c-1b1abe8612f7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:38.115782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:38.131569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35e534-7f58-4b7e-a54a-36f3b1eda0b7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:38.152629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:38.169432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690171c1-6af1-4449-a581-91521a912d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:38.170038 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690171c1-6af1-4449-a581-91521a912d0a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.170319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690171c1-6af1-4449-a581-91521a912d0a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.171496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690171c1-6af1-4449-a581-91521a912d0a None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.171496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690171c1-6af1-4449-a581-91521a912d0a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:38.171659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690171c1-6af1-4449-a581-91521a912d0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:38.173149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690171c1-6af1-4449-a581-91521a912d0a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:38.191187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 271/507] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 11:40:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:38.192103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690171c1-6af1-4449-a581-91521a912d0a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:38.219192 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886197ee-4f68-4bf4-b77c-1b1abe8612f7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:38.253052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275a893-0431-4959-a4f2-1e83d132bda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:38.253648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275a893-0431-4959-a4f2-1e83d132bda5 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.253923 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275a893-0431-4959-a4f2-1e83d132bda5 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.254187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275a893-0431-4959-a4f2-1e83d132bda5 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.254884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275a893-0431-4959-a4f2-1e83d132bda5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:38.255209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275a893-0431-4959-a4f2-1e83d132bda5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:38.256621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275a893-0431-4959-a4f2-1e83d132bda5 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:38.273609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 237/508] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:38.277031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c4e9-0a12-400c-a690-7702d604e8bd None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.277031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c4e9-0a12-400c-a690-7702d604e8bd None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.277031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c4e9-0a12-400c-a690-7702d604e8bd None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.277031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 238/509] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:38.277369 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c4e9-0a12-400c-a690-7702d604e8bd None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:38.298848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690171c1-6af1-4449-a581-91521a912d0a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:38.338524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb30d0e-a182-4412-b4af-a5affb524bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:38.339151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb30d0e-a182-4412-b4af-a5affb524bef None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.339432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb30d0e-a182-4412-b4af-a5affb524bef None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.339702 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb30d0e-a182-4412-b4af-a5affb524bef None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.340381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb30d0e-a182-4412-b4af-a5affb524bef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:38.341830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb30d0e-a182-4412-b4af-a5affb524bef None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:38.388290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:38.393345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c4e9-0a12-400c-a690-7702d604e8bd None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:38.435872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59be85-32f3-4286-ac26-ea41358e8fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:38.436553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59be85-32f3-4286-ac26-ea41358e8fff None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.436829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59be85-32f3-4286-ac26-ea41358e8fff None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.437097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59be85-32f3-4286-ac26-ea41358e8fff None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.437768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59be85-32f3-4286-ac26-ea41358e8fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:38.442420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59be85-32f3-4286-ac26-ea41358e8fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:38.442420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59be85-32f3-4286-ac26-ea41358e8fff None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:38.452403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 239/510] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:38.453260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59be85-32f3-4286-ac26-ea41358e8fff None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:38.454238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:38.460499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 272/511] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:40:38 2022] POST /identity/v3/users => generated 247 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:40:38.461333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb30d0e-a182-4412-b4af-a5affb524bef None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:38.601070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59be85-32f3-4286-ac26-ea41358e8fff None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:38.607739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb30d0e-a182-4412-b4af-a5affb524bef None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:38.646861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdc179-bb05-4f3c-9f8f-c12b5a699953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:38.647377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdc179-bb05-4f3c-9f8f-c12b5a699953 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.648325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdc179-bb05-4f3c-9f8f-c12b5a699953 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.648325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdc179-bb05-4f3c-9f8f-c12b5a699953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.652324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcba3f5-98d8-479c-9e8b-36422059983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:38.652982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcba3f5-98d8-479c-9e8b-36422059983f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.653260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcba3f5-98d8-479c-9e8b-36422059983f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.653535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcba3f5-98d8-479c-9e8b-36422059983f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.660950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcba3f5-98d8-479c-9e8b-36422059983f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:38.662420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcba3f5-98d8-479c-9e8b-36422059983f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-2bcba3f5-98d8-479c-9e8b-36422059983f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:38.676135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:38.677111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:38.677957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:38.677957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:38.677957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:38.677957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:38.677957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:38.677957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:38.677957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:38.677957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:38.677957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 273/512] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:38.678644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dcaf2-2023-4914-8929-2169f1c87a1b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.678933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dcaf2-2023-4914-8929-2169f1c87a1b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.679215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dcaf2-2023-4914-8929-2169f1c87a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.702165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4a3dcaf2-2023-4914-8929-2169f1c87a1b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:38.704485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:38.704630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4a3dcaf2-2023-4914-8929-2169f1c87a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:38.759092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdc179-bb05-4f3c-9f8f-c12b5a699953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:38.761145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdc179-bb05-4f3c-9f8f-c12b5a699953 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:38.766987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:38.781305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 274/513] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:38.784346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dcaf2-2023-4914-8929-2169f1c87a1b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:38.903627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 240/514] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:38.904508 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdc179-bb05-4f3c-9f8f-c12b5a699953 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:38.905616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dcaf2-2023-4914-8929-2169f1c87a1b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:38.957354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d10ec-a307-4d49-82b3-9579802cf9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:38.957999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d10ec-a307-4d49-82b3-9579802cf9d7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:38.958326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d10ec-a307-4d49-82b3-9579802cf9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:38.958605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d10ec-a307-4d49-82b3-9579802cf9d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:38.969751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d10ec-a307-4d49-82b3-9579802cf9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:38.972019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d10ec-a307-4d49-82b3-9579802cf9d7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:38.981712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-602d10ec-a307-4d49-82b3-9579802cf9d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:38.989419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:38.991085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:38.992695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:38.992695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:38.992695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:38.992695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:38.992695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:38.992695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:38.992695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:38.992695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:38.992695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:40:38.992695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:38.996857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 275/515] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 11:40:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:38.997811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d10ec-a307-4d49-82b3-9579802cf9d7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:39.048373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdc179-bb05-4f3c-9f8f-c12b5a699953 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:39.090438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe294bb-b468-4ea3-baa5-03af0be6b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:39.091050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe294bb-b468-4ea3-baa5-03af0be6b4f0 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:39.091359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe294bb-b468-4ea3-baa5-03af0be6b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:39.091636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe294bb-b468-4ea3-baa5-03af0be6b4f0 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:39.092398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe294bb-b468-4ea3-baa5-03af0be6b4f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:39.092707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe294bb-b468-4ea3-baa5-03af0be6b4f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:39.094213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe294bb-b468-4ea3-baa5-03af0be6b4f0 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:39.098940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:39.116246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 241/516] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:39.117784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe294bb-b468-4ea3-baa5-03af0be6b4f0 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:39.189499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d10ec-a307-4d49-82b3-9579802cf9d7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:39.229696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5961a-d16b-4372-865e-a919db541cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:39.230636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5961a-d16b-4372-865e-a919db541cd5 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:39.230912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5961a-d16b-4372-865e-a919db541cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:39.231198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5961a-d16b-4372-865e-a919db541cd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:39.243745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5961a-d16b-4372-865e-a919db541cd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:39.245333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5961a-d16b-4372-865e-a919db541cd5 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-e8e5961a-d16b-4372-865e-a919db541cd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:39.267783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:39.269286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:39.270551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:39.270551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:39.270551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:39.270730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe294bb-b468-4ea3-baa5-03af0be6b4f0 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:39.276712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 276/517] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:39.278335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5961a-d16b-4372-865e-a919db541cd5 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:39.290464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:39.311310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81984a-6e9c-4ded-b97f-79484b627ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:39.311934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81984a-6e9c-4ded-b97f-79484b627ec4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:39.312260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81984a-6e9c-4ded-b97f-79484b627ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:39.312552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81984a-6e9c-4ded-b97f-79484b627ec4 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:39.313360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81984a-6e9c-4ded-b97f-79484b627ec4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:39.313776 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81984a-6e9c-4ded-b97f-79484b627ec4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:39.315293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81984a-6e9c-4ded-b97f-79484b627ec4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:39.362506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 242/518] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 11:40:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:39.363138 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81984a-6e9c-4ded-b97f-79484b627ec4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:39.409724 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:39.414311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5961a-d16b-4372-865e-a919db541cd5 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:39.465684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32386011-7139-4b8a-b41e-4deb495522e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:39.466573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32386011-7139-4b8a-b41e-4deb495522e1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:39.467036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32386011-7139-4b8a-b41e-4deb495522e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:39.467502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32386011-7139-4b8a-b41e-4deb495522e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:39.492946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32386011-7139-4b8a-b41e-4deb495522e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:39.499408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81984a-6e9c-4ded-b97f-79484b627ec4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-32386011-7139-4b8a-b41e-4deb495522e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:39.504253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:39.505193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:39.506025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:39.509941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 277/519] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:39.509941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32386011-7139-4b8a-b41e-4deb495522e1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:39.550411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14a452-cd97-47d8-820f-c7f9a24de280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:39.551015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14a452-cd97-47d8-820f-c7f9a24de280 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:39.551300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14a452-cd97-47d8-820f-c7f9a24de280 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:39.551701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14a452-cd97-47d8-820f-c7f9a24de280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:39.552322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14a452-cd97-47d8-820f-c7f9a24de280 None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=454fb811f6e34e1d881d4948ede058b6) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:39.552631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14a452-cd97-47d8-820f-c7f9a24de280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:39.554108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14a452-cd97-47d8-820f-c7f9a24de280 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:39.567813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 243/520] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:39 2022] GET /identity/v3/role_assignments?user.id=454fb811f6e34e1d881d4948ede058b6&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:39.567813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14a452-cd97-47d8-820f-c7f9a24de280 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:39.628137 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:39.659670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32386011-7139-4b8a-b41e-4deb495522e1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:39.705754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14a452-cd97-47d8-820f-c7f9a24de280 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:39.715999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc98c7-169b-4634-b3be-f06a8348659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:39.716771 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc98c7-169b-4634-b3be-f06a8348659a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:39.717672 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc98c7-169b-4634-b3be-f06a8348659a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:39.718083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc98c7-169b-4634-b3be-f06a8348659a None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:39.719315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc98c7-169b-4634-b3be-f06a8348659a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:39.719724 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc98c7-169b-4634-b3be-f06a8348659a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:39.721627 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc98c7-169b-4634-b3be-f06a8348659a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:39.740483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 278/521] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:39.741558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc98c7-169b-4634-b3be-f06a8348659a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:39.754162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8e55a-d1ca-4215-b261-d739f093376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:39.754847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8e55a-d1ca-4215-b261-d739f093376c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:39.755160 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8e55a-d1ca-4215-b261-d739f093376c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:39.755396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8e55a-d1ca-4215-b261-d739f093376c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:39.774756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:39.856857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8e55a-d1ca-4215-b261-d739f093376c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:39.859215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8e55a-d1ca-4215-b261-d739f093376c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:39.892297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc98c7-169b-4634-b3be-f06a8348659a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:39.911225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:39.935197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a494a0-c503-4262-8a23-c6e490f8e630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:39.935998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a494a0-c503-4262-8a23-c6e490f8e630 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:39.936373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a494a0-c503-4262-8a23-c6e490f8e630 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:39.936679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a494a0-c503-4262-8a23-c6e490f8e630 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:39.937902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a494a0-c503-4262-8a23-c6e490f8e630 None admin] RBAC: Adding query filter params (name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:39.938215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a494a0-c503-4262-8a23-c6e490f8e630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:39.939653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a494a0-c503-4262-8a23-c6e490f8e630 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:39.949445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 279/522] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:39.950503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a494a0-c503-4262-8a23-c6e490f8e630 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:39.989979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 244/523] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:39.990893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8e55a-d1ca-4215-b261-d739f093376c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:40.060445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:40.086653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a494a0-c503-4262-8a23-c6e490f8e630 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:40.103201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8e55a-d1ca-4215-b261-d739f093376c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:40.136105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2517e-c388-49bb-991b-d80382df70f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:40.137124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2517e-c388-49bb-991b-d80382df70f1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.137983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2517e-c388-49bb-991b-d80382df70f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.138340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2517e-c388-49bb-991b-d80382df70f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:40.146124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670cdff-beb4-4336-abbf-5fc32126bf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:40.146752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2517e-c388-49bb-991b-d80382df70f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:40.147102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670cdff-beb4-4336-abbf-5fc32126bf65 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.147248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670cdff-beb4-4336-abbf-5fc32126bf65 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.147595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670cdff-beb4-4336-abbf-5fc32126bf65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:40.148198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2517e-c388-49bb-991b-d80382df70f1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:40.158211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:40.160755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670cdff-beb4-4336-abbf-5fc32126bf65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:40.162264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670cdff-beb4-4336-abbf-5fc32126bf65 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-0ce2517e-c388-49bb-991b-d80382df70f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:40.165471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:40.177066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:40.178061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:40.178061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:40.178061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:40.178061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:40.178061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:40.178061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:40.178061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:40.178061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:40.178061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 280/524] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:40.178061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2517e-c388-49bb-991b-d80382df70f1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-c670cdff-beb4-4336-abbf-5fc32126bf65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:40.179926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:40.180954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:40.182200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:40.182200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:40.182200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:40.182200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 245/525] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:40.182200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670cdff-beb4-4336-abbf-5fc32126bf65 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:40.317862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2517e-c388-49bb-991b-d80382df70f1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:40.353013 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670cdff-beb4-4336-abbf-5fc32126bf65 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:40.361764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1bd984-fffc-4ea6-b4d0-866e86ca1793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:40.362407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bd984-fffc-4ea6-b4d0-866e86ca1793 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.363156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bd984-fffc-4ea6-b4d0-866e86ca1793 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.363462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bd984-fffc-4ea6-b4d0-866e86ca1793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:40.377251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1bd984-fffc-4ea6-b4d0-866e86ca1793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:40.378715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1bd984-fffc-4ea6-b4d0-866e86ca1793 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:40.381322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-bf1bd984-fffc-4ea6-b4d0-866e86ca1793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:40.397003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:40.398639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:40.399514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:40.399514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:40.399514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:40.399514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 281/526] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:40.399514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1bd984-fffc-4ea6-b4d0-866e86ca1793 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:40.402696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374db2d-7c1a-4589-b0f9-711aec5bcab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:40.403836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374db2d-7c1a-4589-b0f9-711aec5bcab2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.405244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374db2d-7c1a-4589-b0f9-711aec5bcab2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.405711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374db2d-7c1a-4589-b0f9-711aec5bcab2 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:40.406634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374db2d-7c1a-4589-b0f9-711aec5bcab2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:40.407093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374db2d-7c1a-4589-b0f9-711aec5bcab2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:40.408769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374db2d-7c1a-4589-b0f9-711aec5bcab2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:40.427057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 246/527] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:40.428897 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374db2d-7c1a-4589-b0f9-711aec5bcab2 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:40.505146 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:40.534118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1bd984-fffc-4ea6-b4d0-866e86ca1793 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:40.574903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374db2d-7c1a-4589-b0f9-711aec5bcab2 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:40.577256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12578e23-6ab5-4001-a368-48995a2b7bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:40.578828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12578e23-6ab5-4001-a368-48995a2b7bc9 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.579221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12578e23-6ab5-4001-a368-48995a2b7bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.579615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12578e23-6ab5-4001-a368-48995a2b7bc9 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:40.580414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12578e23-6ab5-4001-a368-48995a2b7bc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:40.580822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12578e23-6ab5-4001-a368-48995a2b7bc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:40.582305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12578e23-6ab5-4001-a368-48995a2b7bc9 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:40.601328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 282/528] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:40.601328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12578e23-6ab5-4001-a368-48995a2b7bc9 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:40.626037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7065b-d103-4b2f-b839-3b21750d71e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:40.627188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7065b-d103-4b2f-b839-3b21750d71e4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.627649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7065b-d103-4b2f-b839-3b21750d71e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.628071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7065b-d103-4b2f-b839-3b21750d71e4 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:40.629188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7065b-d103-4b2f-b839-3b21750d71e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:40.630074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7065b-d103-4b2f-b839-3b21750d71e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:40.632249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7065b-d103-4b2f-b839-3b21750d71e4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:40.649024 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 247/529] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:40.650384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7065b-d103-4b2f-b839-3b21750d71e4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:40.671564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:40.728630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12578e23-6ab5-4001-a368-48995a2b7bc9 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:40.766103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7065b-d103-4b2f-b839-3b21750d71e4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:40.773095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36373948-f45c-4a96-a7e3-2178421825db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:40.773961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36373948-f45c-4a96-a7e3-2178421825db None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.774381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36373948-f45c-4a96-a7e3-2178421825db None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.774818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36373948-f45c-4a96-a7e3-2178421825db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:40.775733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36373948-f45c-4a96-a7e3-2178421825db None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=496a2e8bf591437888ea82161ca4d5ee) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:40.776277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36373948-f45c-4a96-a7e3-2178421825db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:40.777998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36373948-f45c-4a96-a7e3-2178421825db None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:40.782709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:40.786884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 283/530] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:40 2022] GET /identity/v3/role_assignments?user.id=496a2e8bf591437888ea82161ca4d5ee&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:40.788024 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36373948-f45c-4a96-a7e3-2178421825db None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:40.833157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53455912-1c60-49ec-8628-a49bae5cce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:40.833830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53455912-1c60-49ec-8628-a49bae5cce0f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.834180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53455912-1c60-49ec-8628-a49bae5cce0f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.834527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53455912-1c60-49ec-8628-a49bae5cce0f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:40.850052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53455912-1c60-49ec-8628-a49bae5cce0f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:40.851641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53455912-1c60-49ec-8628-a49bae5cce0f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-53455912-1c60-49ec-8628-a49bae5cce0f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:40.874388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:40.875347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:40.876305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:40.876305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:40.876305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:40.876305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:40.876305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:40.876305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:40.876305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:40.876305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:40.876305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 11:40:40.876305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:40.880720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 248/531] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:40.882812 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fe17f-fd0e-459c-980b-d4452837f7df None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.882812 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fe17f-fd0e-459c-980b-d4452837f7df None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.883057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fe17f-fd0e-459c-980b-d4452837f7df None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:40.886344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 249/532] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:40.887158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fe17f-fd0e-459c-980b-d4452837f7df None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:40.915630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36373948-f45c-4a96-a7e3-2178421825db None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:40.938532 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:40.958965 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9826cef-9387-48db-ba4d-fb276543e24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:40.960441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9826cef-9387-48db-ba4d-fb276543e24d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.960836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9826cef-9387-48db-ba4d-fb276543e24d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.961231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9826cef-9387-48db-ba4d-fb276543e24d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:40.970141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9826cef-9387-48db-ba4d-fb276543e24d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:40.972299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9826cef-9387-48db-ba4d-fb276543e24d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-c9826cef-9387-48db-ba4d-fb276543e24d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:40.988872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:40.990823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:40.993022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:40.993022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:40.993022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:40.993022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:40.993022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:40.993022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:40.993022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:40.993022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:40.993022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 284/533] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:40.994682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de92ac8-5f6a-4a42-8b98-c49875c693e3 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:40.994682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de92ac8-5f6a-4a42-8b98-c49875c693e3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:40.995552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de92ac8-5f6a-4a42-8b98-c49875c693e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:41.017171 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-7de92ac8-5f6a-4a42-8b98-c49875c693e3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:41.018452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7de92ac8-5f6a-4a42-8b98-c49875c693e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:41.063857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99fe17f-fd0e-459c-980b-d4452837f7df None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:41.107449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5d711-9616-4230-a6a1-138a49cd6b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:41.108444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5d711-9616-4230-a6a1-138a49cd6b9c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:41.108983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5d711-9616-4230-a6a1-138a49cd6b9c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:41.109436 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5d711-9616-4230-a6a1-138a49cd6b9c None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:41.110409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5d711-9616-4230-a6a1-138a49cd6b9c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:41.110902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5d711-9616-4230-a6a1-138a49cd6b9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:41.112744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5d711-9616-4230-a6a1-138a49cd6b9c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:41.133405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 285/534] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:41.134859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de92ac8-5f6a-4a42-8b98-c49875c693e3 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:41.141835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 250/535] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Oct 4 11:40:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:41.143619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5d711-9616-4230-a6a1-138a49cd6b9c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:41.179275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:41.277735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5d711-9616-4230-a6a1-138a49cd6b9c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:41.289009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de92ac8-5f6a-4a42-8b98-c49875c693e3 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:41.312621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:41.345489 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646a9ea-f2fa-42e4-8f02-0d1e37324035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:41.346101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646a9ea-f2fa-42e4-8f02-0d1e37324035 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:41.346384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646a9ea-f2fa-42e4-8f02-0d1e37324035 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:41.346663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646a9ea-f2fa-42e4-8f02-0d1e37324035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:41.362254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423266d-ee04-4002-8e28-2f1e455b502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:41.362780 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423266d-ee04-4002-8e28-2f1e455b502e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:41.363065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423266d-ee04-4002-8e28-2f1e455b502e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:41.363339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423266d-ee04-4002-8e28-2f1e455b502e None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:41.364043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423266d-ee04-4002-8e28-2f1e455b502e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:41.364374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423266d-ee04-4002-8e28-2f1e455b502e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:41.365820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423266d-ee04-4002-8e28-2f1e455b502e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:41.404040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 286/536] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:41.410310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423266d-ee04-4002-8e28-2f1e455b502e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:41.453925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646a9ea-f2fa-42e4-8f02-0d1e37324035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:41.455420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646a9ea-f2fa-42e4-8f02-0d1e37324035 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:41.496531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:41.560880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423266d-ee04-4002-8e28-2f1e455b502e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:41.572457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 251/537] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 11:40:41.573982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646a9ea-f2fa-42e4-8f02-0d1e37324035 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:41.596574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8b020-23d8-46b0-ae0e-456699f157c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:41.597243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8b020-23d8-46b0-ae0e-456699f157c2 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:41.597524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8b020-23d8-46b0-ae0e-456699f157c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:41.597854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8b020-23d8-46b0-ae0e-456699f157c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:41.598589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8b020-23d8-46b0-ae0e-456699f157c2 None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=ed14c1bc105d4c5cade5813edc1f0dac) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:41.598898 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8b020-23d8-46b0-ae0e-456699f157c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:41.600383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8b020-23d8-46b0-ae0e-456699f157c2 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:41.607029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 287/538] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:41 2022] GET /identity/v3/role_assignments?user.id=ed14c1bc105d4c5cade5813edc1f0dac&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:41.608120 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8b020-23d8-46b0-ae0e-456699f157c2 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:41.643476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:41.719204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8b020-23d8-46b0-ae0e-456699f157c2 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:41.756398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9a76b-2fcc-458e-a43b-f72af356c3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:41.757242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9a76b-2fcc-458e-a43b-f72af356c3cd None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:41.757623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9a76b-2fcc-458e-a43b-f72af356c3cd None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:41.757998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9a76b-2fcc-458e-a43b-f72af356c3cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:41.770991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9a76b-2fcc-458e-a43b-f72af356c3cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:41.772441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646a9ea-f2fa-42e4-8f02-0d1e37324035 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:41.772791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9a76b-2fcc-458e-a43b-f72af356c3cd None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-56c9a76b-2fcc-458e-a43b-f72af356c3cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:41.790304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:41.791320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:41.792809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:41.792809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:41.792809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:41.792809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 288/539] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:41.797724 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9a76b-2fcc-458e-a43b-f72af356c3cd None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:41.849070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:41.865191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3159144-3184-4443-8e48-f4adbc1bc4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:41.868199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3159144-3184-4443-8e48-f4adbc1bc4eb None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:41.868199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3159144-3184-4443-8e48-f4adbc1bc4eb None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:41.868199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3159144-3184-4443-8e48-f4adbc1bc4eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:41.885558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3159144-3184-4443-8e48-f4adbc1bc4eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:41.904641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-a3159144-3184-4443-8e48-f4adbc1bc4eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:41.905066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:41.907068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:41.909043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 252/540] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:41.916227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3159144-3184-4443-8e48-f4adbc1bc4eb None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:41.918168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9a76b-2fcc-458e-a43b-f72af356c3cd None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:41.964597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977802d-7ca6-4ddc-963d-536eaad8da1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:41.965359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977802d-7ca6-4ddc-963d-536eaad8da1d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:41.965643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977802d-7ca6-4ddc-963d-536eaad8da1d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:41.965919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977802d-7ca6-4ddc-963d-536eaad8da1d None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:41.966598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977802d-7ca6-4ddc-963d-536eaad8da1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:41.966905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977802d-7ca6-4ddc-963d-536eaad8da1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:41.968749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1977802d-7ca6-4ddc-963d-536eaad8da1d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:41.987046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 289/541] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:41.992810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977802d-7ca6-4ddc-963d-536eaad8da1d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:42.106609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1977802d-7ca6-4ddc-963d-536eaad8da1d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:42.107825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3159144-3184-4443-8e48-f4adbc1bc4eb None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:42.151152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423067b-cc82-4c83-a132-cfd8c002ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:42.152253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423067b-cc82-4c83-a132-cfd8c002ec04 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:42.152543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423067b-cc82-4c83-a132-cfd8c002ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:42.152824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423067b-cc82-4c83-a132-cfd8c002ec04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:42.157405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f418-5ba4-478d-8c80-54aeb0e8b859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:42.158275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f418-5ba4-478d-8c80-54aeb0e8b859 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:42.158767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f418-5ba4-478d-8c80-54aeb0e8b859 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:42.159257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f418-5ba4-478d-8c80-54aeb0e8b859 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:42.160252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886f418-5ba4-478d-8c80-54aeb0e8b859 None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:42.160824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886f418-5ba4-478d-8c80-54aeb0e8b859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:42.161209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423067b-cc82-4c83-a132-cfd8c002ec04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:42.162657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886f418-5ba4-478d-8c80-54aeb0e8b859 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:42.162768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423067b-cc82-4c83-a132-cfd8c002ec04 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:42.172620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 253/542] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:42.174230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb766d-0940-4b34-9118-7f4cd9f9a8e1 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:42.174718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb766d-0940-4b34-9118-7f4cd9f9a8e1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:42.175213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb766d-0940-4b34-9118-7f4cd9f9a8e1 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-f423067b-cc82-4c83-a132-cfd8c002ec04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:42.176029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:42.176875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:42.177714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:42.177714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:42.177714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:42.177714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:42.177714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:42.177714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:42.177714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:42.177714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:42.178111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 290/543] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:42.178707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423067b-cc82-4c83-a132-cfd8c002ec04 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:42.179796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 254/544] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:42.183673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb766d-0940-4b34-9118-7f4cd9f9a8e1 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:42.190396 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:42.267928 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:42.307670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423067b-cc82-4c83-a132-cfd8c002ec04 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:42.335472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb766d-0940-4b34-9118-7f4cd9f9a8e1 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:42.345863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de011c63-a88d-421b-bd55-43638758b39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:42.346595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de011c63-a88d-421b-bd55-43638758b39f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:42.346974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de011c63-a88d-421b-bd55-43638758b39f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:42.347367 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de011c63-a88d-421b-bd55-43638758b39f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:42.360299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de011c63-a88d-421b-bd55-43638758b39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:42.361987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de011c63-a88d-421b-bd55-43638758b39f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-de011c63-a88d-421b-bd55-43638758b39f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:42.378113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:42.379804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:42.380912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:42.380912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:42.380912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:42.382367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1136c-ba56-4d40-bd83-96848298d3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:42.383629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1136c-ba56-4d40-bd83-96848298d3c8 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:42.384120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1136c-ba56-4d40-bd83-96848298d3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:42.384554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1136c-ba56-4d40-bd83-96848298d3c8 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:42.385939 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1136c-ba56-4d40-bd83-96848298d3c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:42.386484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1136c-ba56-4d40-bd83-96848298d3c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:42.386948 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 291/545] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:42.388354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1136c-ba56-4d40-bd83-96848298d3c8 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:42.388519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979252f-62db-4494-82b7-2701b2b2691b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:42.388958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979252f-62db-4494-82b7-2701b2b2691b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:42.389341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979252f-62db-4494-82b7-2701b2b2691b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:42.404199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 255/546] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:42.405888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1136c-ba56-4d40-bd83-96848298d3c8 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:42.412144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1979252f-62db-4494-82b7-2701b2b2691b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:42.413724 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1979252f-62db-4494-82b7-2701b2b2691b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:42.474645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:42.504087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 292/547] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:42.504087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979252f-62db-4494-82b7-2701b2b2691b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:42.625569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1136c-ba56-4d40-bd83-96848298d3c8 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:42.628290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979252f-62db-4494-82b7-2701b2b2691b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:42.651542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:42.669775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd911d2-4a3e-4426-9011-6e142b7b7280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:42.670867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd911d2-4a3e-4426-9011-6e142b7b7280 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:42.671347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd911d2-4a3e-4426-9011-6e142b7b7280 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:42.671820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd911d2-4a3e-4426-9011-6e142b7b7280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:42.672818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd911d2-4a3e-4426-9011-6e142b7b7280 None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=073e43c049224af29e9f6d426ea47e84) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:42.673328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd911d2-4a3e-4426-9011-6e142b7b7280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:42.674962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd911d2-4a3e-4426-9011-6e142b7b7280 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:42.682826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 293/548] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:42 2022] GET /identity/v3/role_assignments?user.id=073e43c049224af29e9f6d426ea47e84&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:42.703310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd230-fdb1-4ef4-afa4-2b133329b079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:42.704376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd230-fdb1-4ef4-afa4-2b133329b079 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:42.704861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd230-fdb1-4ef4-afa4-2b133329b079 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:42.705333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd230-fdb1-4ef4-afa4-2b133329b079 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:42.706241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd230-fdb1-4ef4-afa4-2b133329b079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:42.706753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd230-fdb1-4ef4-afa4-2b133329b079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:42.708559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd230-fdb1-4ef4-afa4-2b133329b079 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:42.731315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 256/549] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:42.737391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd911d2-4a3e-4426-9011-6e142b7b7280 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:42.835185 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:42.854334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd230-fdb1-4ef4-afa4-2b133329b079 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:42.945703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd911d2-4a3e-4426-9011-6e142b7b7280 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:42.999653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02ba2e-bea4-43e4-892d-c1e86c068810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:43.002950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02ba2e-bea4-43e4-892d-c1e86c068810 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:43.004308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02ba2e-bea4-43e4-892d-c1e86c068810 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:43.004588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02ba2e-bea4-43e4-892d-c1e86c068810 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:43.024843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02ba2e-bea4-43e4-892d-c1e86c068810 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:43.026405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02ba2e-bea4-43e4-892d-c1e86c068810 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-ab02ba2e-bea4-43e4-892d-c1e86c068810 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:43.053287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:43.054461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:43.056206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 294/550] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:40:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:43.065938 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02ba2e-bea4-43e4-892d-c1e86c068810 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:43.116198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd230-fdb1-4ef4-afa4-2b133329b079 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:43.233880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce74ff-2989-4f4a-bfc5-06bf04cfcfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:43.234511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce74ff-2989-4f4a-bfc5-06bf04cfcfe9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:43.234872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce74ff-2989-4f4a-bfc5-06bf04cfcfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:43.237128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce74ff-2989-4f4a-bfc5-06bf04cfcfe9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:43.259451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce74ff-2989-4f4a-bfc5-06bf04cfcfe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-c3ce74ff-2989-4f4a-bfc5-06bf04cfcfe9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:43.271058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:43.272061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:43.273376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 257/551] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:43.283499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce74ff-2989-4f4a-bfc5-06bf04cfcfe9 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:43.377566 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:43.393067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02ba2e-bea4-43e4-892d-c1e86c068810 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:43.420387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:43.469893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31973a73-9b8e-453a-a098-93a563d0a735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:43.470506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31973a73-9b8e-453a-a098-93a563d0a735 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:43.470783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31973a73-9b8e-453a-a098-93a563d0a735 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:43.471052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31973a73-9b8e-453a-a098-93a563d0a735 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:43.471782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31973a73-9b8e-453a-a098-93a563d0a735 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:43.472083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31973a73-9b8e-453a-a098-93a563d0a735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:43.477796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31973a73-9b8e-453a-a098-93a563d0a735 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:43.502101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 295/552] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 11:40:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:43.503221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab51b1-95fb-4903-b83e-3c7600593265 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:43.503445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab51b1-95fb-4903-b83e-3c7600593265 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:43.503727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab51b1-95fb-4903-b83e-3c7600593265 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:43.509025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 296/553] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:43.509025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab51b1-95fb-4903-b83e-3c7600593265 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:43.518387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce74ff-2989-4f4a-bfc5-06bf04cfcfe9 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:43.565696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b600fdf-73a1-49c1-ad7f-ad082c4986bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:43.566967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b600fdf-73a1-49c1-ad7f-ad082c4986bb None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:43.567694 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b600fdf-73a1-49c1-ad7f-ad082c4986bb None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:43.568115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b600fdf-73a1-49c1-ad7f-ad082c4986bb None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:43.569201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b600fdf-73a1-49c1-ad7f-ad082c4986bb None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:43.569652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b600fdf-73a1-49c1-ad7f-ad082c4986bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:43.571579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b600fdf-73a1-49c1-ad7f-ad082c4986bb None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:43.583719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 258/554] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:43.585269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0f946-0e66-43cd-ab7f-f12df3dd6df1 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:43.585685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0f946-0e66-43cd-ab7f-f12df3dd6df1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:43.586086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0f946-0e66-43cd-ab7f-f12df3dd6df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:43.610347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-0ef0f946-0e66-43cd-ab7f-f12df3dd6df1 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:43.611978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0ef0f946-0e66-43cd-ab7f-f12df3dd6df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:43.640958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab51b1-95fb-4903-b83e-3c7600593265 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:43.684563 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6230e9d-b4fb-4bdb-b2f4-f273c3f31ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:43.685431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6230e9d-b4fb-4bdb-b2f4-f273c3f31ed5 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:43.685855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6230e9d-b4fb-4bdb-b2f4-f273c3f31ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:43.686347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6230e9d-b4fb-4bdb-b2f4-f273c3f31ed5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:43.688251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 259/555] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:43.688251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0f946-0e66-43cd-ab7f-f12df3dd6df1 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:43.735744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:43.747901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:43.790104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6230e9d-b4fb-4bdb-b2f4-f273c3f31ed5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:43.791624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6230e9d-b4fb-4bdb-b2f4-f273c3f31ed5 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:43.898475 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0f946-0e66-43cd-ab7f-f12df3dd6df1 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:43.950173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 297/556] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:43.958542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6230e9d-b4fb-4bdb-b2f4-f273c3f31ed5 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:43.976648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b9fca-c4fd-4a93-9fd6-40cf6ebbd838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:43.977563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b9fca-c4fd-4a93-9fd6-40cf6ebbd838 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:43.977991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b9fca-c4fd-4a93-9fd6-40cf6ebbd838 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:43.978434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b9fca-c4fd-4a93-9fd6-40cf6ebbd838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:43.999151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b9fca-c4fd-4a93-9fd6-40cf6ebbd838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:44.000928 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b9fca-c4fd-4a93-9fd6-40cf6ebbd838 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-2d9b9fca-c4fd-4a93-9fd6-40cf6ebbd838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:44.033742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:44.034920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:44.036032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:44.036032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:44.036032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:44.039119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 260/557] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:44.045464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b9fca-c4fd-4a93-9fd6-40cf6ebbd838 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:44.080620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:44.127145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6230e9d-b4fb-4bdb-b2f4-f273c3f31ed5 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:44.140611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:44.162050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b9fca-c4fd-4a93-9fd6-40cf6ebbd838 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:44.180707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d421c-02c1-4195-86e4-014c3cf3bd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:44.181634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d421c-02c1-4195-86e4-014c3cf3bd22 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.182659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d421c-02c1-4195-86e4-014c3cf3bd22 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.183092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d421c-02c1-4195-86e4-014c3cf3bd22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.197663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d421c-02c1-4195-86e4-014c3cf3bd22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:44.200871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d421c-02c1-4195-86e4-014c3cf3bd22 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:44.207527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1efe2-0433-4aca-9d19-367998e853ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:44.208405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1efe2-0433-4aca-9d19-367998e853ac None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.209233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1efe2-0433-4aca-9d19-367998e853ac None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.209805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1efe2-0433-4aca-9d19-367998e853ac None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.210697 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1efe2-0433-4aca-9d19-367998e853ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:44.211217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1efe2-0433-4aca-9d19-367998e853ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:44.213107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1efe2-0433-4aca-9d19-367998e853ac None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-923d421c-02c1-4195-86e4-014c3cf3bd22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:44.219400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:44.220424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:44.221329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:44.221329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:44.221329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:44.223470 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 298/558] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:44.224565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d421c-02c1-4195-86e4-014c3cf3bd22 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:44.235269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 261/559] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:44.235269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1efe2-0433-4aca-9d19-367998e853ac None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:44.334993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d421c-02c1-4195-86e4-014c3cf3bd22 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:44.337510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1efe2-0433-4aca-9d19-367998e853ac None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:44.358638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:44.374489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6df247-a12f-4e2d-8a6f-611ea9dd7c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:44.375120 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6df247-a12f-4e2d-8a6f-611ea9dd7c06 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.377831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6df247-a12f-4e2d-8a6f-611ea9dd7c06 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.377938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4378404-7470-475b-ab28-2c965d76a252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:44.378031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6df247-a12f-4e2d-8a6f-611ea9dd7c06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.378105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4378404-7470-475b-ab28-2c965d76a252 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.378105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4378404-7470-475b-ab28-2c965d76a252 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.378105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4378404-7470-475b-ab28-2c965d76a252 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.378105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4378404-7470-475b-ab28-2c965d76a252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:44.378105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4378404-7470-475b-ab28-2c965d76a252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:44.379280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4378404-7470-475b-ab28-2c965d76a252 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:44.381842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:44.389660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6df247-a12f-4e2d-8a6f-611ea9dd7c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:44.400452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 262/560] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:44.400452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4378404-7470-475b-ab28-2c965d76a252 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-2c6df247-a12f-4e2d-8a6f-611ea9dd7c06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:44.401236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:44.402531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 299/561] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:44.403381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6023c-6b31-4bea-af5d-463eb72b56c7 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.405170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6023c-6b31-4bea-af5d-463eb72b56c7 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.405170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6023c-6b31-4bea-af5d-463eb72b56c7 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.405170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 300/562] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:44.405170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6023c-6b31-4bea-af5d-463eb72b56c7 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:44.516614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4378404-7470-475b-ab28-2c965d76a252 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:44.516817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6023c-6b31-4bea-af5d-463eb72b56c7 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:44.554306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9e437-46e0-418f-87f8-7e9d11cde807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:44.554518 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b79b9-199a-4a4f-aa90-a9ae73645bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:44.554942 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e437-46e0-418f-87f8-7e9d11cde807 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.555147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b79b9-199a-4a4f-aa90-a9ae73645bba None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.555248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e437-46e0-418f-87f8-7e9d11cde807 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.555429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b79b9-199a-4a4f-aa90-a9ae73645bba None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.555525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e437-46e0-418f-87f8-7e9d11cde807 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.555704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b79b9-199a-4a4f-aa90-a9ae73645bba None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.563349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b79b9-199a-4a4f-aa90-a9ae73645bba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:44.563535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9e437-46e0-418f-87f8-7e9d11cde807 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:44.564850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b79b9-199a-4a4f-aa90-a9ae73645bba None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:44.565021 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9e437-46e0-418f-87f8-7e9d11cde807 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-43c9e437-46e0-418f-87f8-7e9d11cde807 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:44.576973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:44.578009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:44.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:44.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:44.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:44.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:44.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:44.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:44.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:44.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:44.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 11:40:44.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-8b4b79b9-199a-4a4f-aa90-a9ae73645bba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:44.579540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:44.581138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:44.582945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:44.582945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:44.582945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:44.582945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:44.582945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:44.582945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:44.582945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:44.582945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:44.583900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 263/563] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:44.584036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 301/564] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:44.584036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b79b9-199a-4a4f-aa90-a9ae73645bba None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:44.588363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9e889-b628-457f-8aa0-e0d4cb07a4be None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.588572 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9e889-b628-457f-8aa0-e0d4cb07a4be None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.588852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9e889-b628-457f-8aa0-e0d4cb07a4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.607980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-c1d9e889-b628-457f-8aa0-e0d4cb07a4be None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:44.609200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c1d9e889-b628-457f-8aa0-e0d4cb07a4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:44.627240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:44.673152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:44.698059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 264/565] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:44.698059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9e889-b628-457f-8aa0-e0d4cb07a4be None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:44.703238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b79b9-199a-4a4f-aa90-a9ae73645bba None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:44.753624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7e956-c142-4a58-8043-161d2bd5d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:44.754390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7e956-c142-4a58-8043-161d2bd5d1e9 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.754921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7e956-c142-4a58-8043-161d2bd5d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.755196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7e956-c142-4a58-8043-161d2bd5d1e9 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.756308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7e956-c142-4a58-8043-161d2bd5d1e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:44.756620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7e956-c142-4a58-8043-161d2bd5d1e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:44.757995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7e956-c142-4a58-8043-161d2bd5d1e9 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:44.771457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 302/566] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:44.772456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7e956-c142-4a58-8043-161d2bd5d1e9 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:44.817923 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9e889-b628-457f-8aa0-e0d4cb07a4be None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:44.856079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e909c-4681-4e99-95df-8c03eb2e526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:44.856805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e909c-4681-4e99-95df-8c03eb2e526f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.857098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e909c-4681-4e99-95df-8c03eb2e526f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.857620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e909c-4681-4e99-95df-8c03eb2e526f None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.858450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e909c-4681-4e99-95df-8c03eb2e526f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:44.858761 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e909c-4681-4e99-95df-8c03eb2e526f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:44.860216 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e909c-4681-4e99-95df-8c03eb2e526f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:44.886912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 265/567] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 11:40:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:44.887911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e909c-4681-4e99-95df-8c03eb2e526f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:44.929546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7e956-c142-4a58-8043-161d2bd5d1e9 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:44.931297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:44.951375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:44.968830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef26a9-d991-4479-b3ca-7f4757bd54a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:44.969443 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef26a9-d991-4479-b3ca-7f4757bd54a5 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:44.969723 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef26a9-d991-4479-b3ca-7f4757bd54a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:44.969994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef26a9-d991-4479-b3ca-7f4757bd54a5 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:44.970695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef26a9-d991-4479-b3ca-7f4757bd54a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:44.971056 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef26a9-d991-4479-b3ca-7f4757bd54a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:44.972796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef26a9-d991-4479-b3ca-7f4757bd54a5 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:44.987240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 303/568] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:44.988095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef26a9-d991-4479-b3ca-7f4757bd54a5 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:45.019182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e909c-4681-4e99-95df-8c03eb2e526f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.052757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2581cab-98d5-41a3-a75f-ceaa8db3ab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:45.053504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2581cab-98d5-41a3-a75f-ceaa8db3ab7b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.053782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2581cab-98d5-41a3-a75f-ceaa8db3ab7b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.054055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2581cab-98d5-41a3-a75f-ceaa8db3ab7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.064625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2581cab-98d5-41a3-a75f-ceaa8db3ab7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:45.066079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2581cab-98d5-41a3-a75f-ceaa8db3ab7b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-e2581cab-98d5-41a3-a75f-ceaa8db3ab7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:45.081208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:45.082332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:45.083221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:45.083221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:45.083221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:45.083221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 266/569] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:45.083221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2581cab-98d5-41a3-a75f-ceaa8db3ab7b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:45.095870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef26a9-d991-4479-b3ca-7f4757bd54a5 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.129906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df980a-6814-47e0-878e-02e10e49a751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:45.130446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df980a-6814-47e0-878e-02e10e49a751 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.130725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df980a-6814-47e0-878e-02e10e49a751 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.130998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df980a-6814-47e0-878e-02e10e49a751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.141985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df980a-6814-47e0-878e-02e10e49a751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:45.145506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df980a-6814-47e0-878e-02e10e49a751 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-27df980a-6814-47e0-878e-02e10e49a751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:45.158153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:45.159180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:45.160076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:45.160076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:45.160076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:45.160076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 304/570] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:45.160076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df980a-6814-47e0-878e-02e10e49a751 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:45.176813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:45.181585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:45.187912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2581cab-98d5-41a3-a75f-ceaa8db3ab7b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.221079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4889a-06fc-4e88-843b-5c69f036559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:45.221707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4889a-06fc-4e88-843b-5c69f036559f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.222071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4889a-06fc-4e88-843b-5c69f036559f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.222271 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4889a-06fc-4e88-843b-5c69f036559f None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.233925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4889a-06fc-4e88-843b-5c69f036559f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-cab4889a-06fc-4e88-843b-5c69f036559f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:45.238739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:45.240491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 267/571] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:45.241511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2f658-038a-4d4c-a6c0-8cb005bb70ed None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.242427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2f658-038a-4d4c-a6c0-8cb005bb70ed None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.242427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2f658-038a-4d4c-a6c0-8cb005bb70ed None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.243025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 268/572] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:45.243584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2f658-038a-4d4c-a6c0-8cb005bb70ed None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:45.273347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df980a-6814-47e0-878e-02e10e49a751 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.310663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa28a4-c813-4867-87a5-6c18fbea8cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:45.311358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa28a4-c813-4867-87a5-6c18fbea8cba None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.311647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa28a4-c813-4867-87a5-6c18fbea8cba None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.311921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa28a4-c813-4867-87a5-6c18fbea8cba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.312686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa28a4-c813-4867-87a5-6c18fbea8cba None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=734d8f03ff2949959cbfce98212a3ac6) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:45.312994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa28a4-c813-4867-87a5-6c18fbea8cba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:45.314429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa28a4-c813-4867-87a5-6c18fbea8cba None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:45.320703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 305/573] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:45 2022] GET /identity/v3/role_assignments?user.id=734d8f03ff2949959cbfce98212a3ac6&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:45.321510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa28a4-c813-4867-87a5-6c18fbea8cba None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:45.376832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2f658-038a-4d4c-a6c0-8cb005bb70ed None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.418463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38849a9c-f463-4dcc-8a31-faaa213f990f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:45.418997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38849a9c-f463-4dcc-8a31-faaa213f990f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.419278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38849a9c-f463-4dcc-8a31-faaa213f990f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.419552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38849a9c-f463-4dcc-8a31-faaa213f990f None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.420233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38849a9c-f463-4dcc-8a31-faaa213f990f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:45.420567 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38849a9c-f463-4dcc-8a31-faaa213f990f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:45.421997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38849a9c-f463-4dcc-8a31-faaa213f990f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:45.424477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:45.436619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 269/574] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:45.438431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38849a9c-f463-4dcc-8a31-faaa213f990f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:45.452589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa28a4-c813-4867-87a5-6c18fbea8cba None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.462852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:45.492477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cdee9-e4ae-4294-9a9b-6a1d6047284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:45.493106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cdee9-e4ae-4294-9a9b-6a1d6047284d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.493455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cdee9-e4ae-4294-9a9b-6a1d6047284d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.496113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cdee9-e4ae-4294-9a9b-6a1d6047284d None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.496113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cdee9-e4ae-4294-9a9b-6a1d6047284d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:45.496113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cdee9-e4ae-4294-9a9b-6a1d6047284d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:45.496113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cdee9-e4ae-4294-9a9b-6a1d6047284d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:45.509320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 306/575] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:45.511091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59ede4-e2bb-406b-a3b7-ad91bf353792 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.511376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59ede4-e2bb-406b-a3b7-ad91bf353792 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.511651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59ede4-e2bb-406b-a3b7-ad91bf353792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.542210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4c59ede4-e2bb-406b-a3b7-ad91bf353792 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:45.543524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4c59ede4-e2bb-406b-a3b7-ad91bf353792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:45.562451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38849a9c-f463-4dcc-8a31-faaa213f990f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.605076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b0f8-1537-497a-a0f5-47ef77e10952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:45.605981 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b0f8-1537-497a-a0f5-47ef77e10952 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.606425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b0f8-1537-497a-a0f5-47ef77e10952 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.606848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b0f8-1537-497a-a0f5-47ef77e10952 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.607930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71b0f8-1537-497a-a0f5-47ef77e10952 None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:45.608436 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71b0f8-1537-497a-a0f5-47ef77e10952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:45.610480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71b0f8-1537-497a-a0f5-47ef77e10952 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:45.619844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 270/576] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:45.620707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b0f8-1537-497a-a0f5-47ef77e10952 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:45.628375 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 307/577] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:45.629470 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59ede4-e2bb-406b-a3b7-ad91bf353792 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:45.713049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:45.744691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:45.749992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59ede4-e2bb-406b-a3b7-ad91bf353792 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.774375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b0f8-1537-497a-a0f5-47ef77e10952 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.792400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422fcb5-3132-44b0-ab71-1b1aff0cd095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:45.793050 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422fcb5-3132-44b0-ab71-1b1aff0cd095 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.793365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422fcb5-3132-44b0-ab71-1b1aff0cd095 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.793673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422fcb5-3132-44b0-ab71-1b1aff0cd095 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.806311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422fcb5-3132-44b0-ab71-1b1aff0cd095 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:45.808399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422fcb5-3132-44b0-ab71-1b1aff0cd095 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:45.819845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181dd8b-61e2-4f93-84b8-a67aa8f66377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:45.820528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181dd8b-61e2-4f93-84b8-a67aa8f66377 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-c422fcb5-3132-44b0-ab71-1b1aff0cd095 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:45.820695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:45.821700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:45.823324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:45.823324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:45.823324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:45.823324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:45.823324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:45.823324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:45.823324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:45.823324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:45.823755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181dd8b-61e2-4f93-84b8-a67aa8f66377 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.823755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181dd8b-61e2-4f93-84b8-a67aa8f66377 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.823883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 308/578] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:45.823883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422fcb5-3132-44b0-ab71-1b1aff0cd095 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:45.836757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181dd8b-61e2-4f93-84b8-a67aa8f66377 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:45.839134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181dd8b-61e2-4f93-84b8-a67aa8f66377 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-9181dd8b-61e2-4f93-84b8-a67aa8f66377 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:45.854906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:45.856041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:45.857422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:45.857422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:45.857422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:45.857422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:45.857422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:45.857422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:45.857422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:45.857422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:45.857422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 11:40:45.857422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:45.857980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 271/579] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:45.858264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181dd8b-61e2-4f93-84b8-a67aa8f66377 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:45.945938 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422fcb5-3132-44b0-ab71-1b1aff0cd095 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.975155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:45.985391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3347f2d-79e3-4766-9449-3020f9bf17da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:45.985763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181dd8b-61e2-4f93-84b8-a67aa8f66377 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:45.986412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347f2d-79e3-4766-9449-3020f9bf17da None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:45.986412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347f2d-79e3-4766-9449-3020f9bf17da None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:45.989630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347f2d-79e3-4766-9449-3020f9bf17da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:45.999886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3347f2d-79e3-4766-9449-3020f9bf17da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:46.002094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3347f2d-79e3-4766-9449-3020f9bf17da None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-d3347f2d-79e3-4766-9449-3020f9bf17da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:46.019645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:46.020863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:46.021949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:46.021949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:46.021949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:46.021949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 309/580] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:46.021949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3347f2d-79e3-4766-9449-3020f9bf17da None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:46.027987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660d470-8fec-467f-9638-11011bcd1b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:46.028843 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660d470-8fec-467f-9638-11011bcd1b13 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:46.029120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660d470-8fec-467f-9638-11011bcd1b13 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:46.029390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660d470-8fec-467f-9638-11011bcd1b13 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:46.030081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660d470-8fec-467f-9638-11011bcd1b13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:46.030392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660d470-8fec-467f-9638-11011bcd1b13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:46.031835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660d470-8fec-467f-9638-11011bcd1b13 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:46.040401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:46.051307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 272/581] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:46.051307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660d470-8fec-467f-9638-11011bcd1b13 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:46.147537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3347f2d-79e3-4766-9449-3020f9bf17da None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:46.165734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660d470-8fec-467f-9638-11011bcd1b13 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:46.185452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3258-c372-4856-8a21-ee474e441274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:46.186061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3258-c372-4856-8a21-ee474e441274 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:46.186336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3258-c372-4856-8a21-ee474e441274 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:46.186606 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3258-c372-4856-8a21-ee474e441274 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:46.187333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef3258-c372-4856-8a21-ee474e441274 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:46.187642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef3258-c372-4856-8a21-ee474e441274 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:46.189147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef3258-c372-4856-8a21-ee474e441274 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:46.207162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 310/582] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 11:40:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:46.208106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3258-c372-4856-8a21-ee474e441274 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:46.255632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:46.257852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076195c-19dd-40be-88f6-87c96470a43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:46.259447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076195c-19dd-40be-88f6-87c96470a43f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:46.259957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076195c-19dd-40be-88f6-87c96470a43f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:46.260488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076195c-19dd-40be-88f6-87c96470a43f None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:46.284670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076195c-19dd-40be-88f6-87c96470a43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-f076195c-19dd-40be-88f6-87c96470a43f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:46.290010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:46.290961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:46.292537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 273/583] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:46.293309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076195c-19dd-40be-88f6-87c96470a43f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:46.328896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3258-c372-4856-8a21-ee474e441274 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:46.356854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:46.368313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3422e3b-5c9d-4043-9f13-a7b702ad40a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:46.369126 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3422e3b-5c9d-4043-9f13-a7b702ad40a6 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:46.369493 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3422e3b-5c9d-4043-9f13-a7b702ad40a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:46.369964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3422e3b-5c9d-4043-9f13-a7b702ad40a6 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:46.379092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3422e3b-5c9d-4043-9f13-a7b702ad40a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:46.379092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3422e3b-5c9d-4043-9f13-a7b702ad40a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:46.379092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3422e3b-5c9d-4043-9f13-a7b702ad40a6 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:46.385373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 311/584] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:46.386295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3422e3b-5c9d-4043-9f13-a7b702ad40a6 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:46.402219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076195c-19dd-40be-88f6-87c96470a43f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:46.438340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0f6dc-0048-42d5-a106-a7a3029d6ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:46.438992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f6dc-0048-42d5-a106-a7a3029d6ff1 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:46.439266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f6dc-0048-42d5-a106-a7a3029d6ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:46.439538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f6dc-0048-42d5-a106-a7a3029d6ff1 None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/454fb811f6e34e1d881d4948ede058b6/roles/5b919ba5900449358b3277530a7eff6c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:46.455220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0f6dc-0048-42d5-a106-a7a3029d6ff1 None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=454fb811f6e34e1d881d4948ede058b6, role_id=5b919ba5900449358b3277530a7eff6c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:46.456924 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0f6dc-0048-42d5-a106-a7a3029d6ff1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:46.489770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:46.493478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3422e3b-5c9d-4043-9f13-a7b702ad40a6 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:46.528025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b0ce1-1a8f-43fc-a35d-e26565c5d0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:46.528642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b0ce1-1a8f-43fc-a35d-e26565c5d0ca None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:46.528928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b0ce1-1a8f-43fc-a35d-e26565c5d0ca None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:46.529207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b0ce1-1a8f-43fc-a35d-e26565c5d0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:46.541910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b0ce1-1a8f-43fc-a35d-e26565c5d0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:46.543479 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b0ce1-1a8f-43fc-a35d-e26565c5d0ca None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:46.548484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 274/585] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:40:46 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/454fb811f6e34e1d881d4948ede058b6/roles/5b919ba5900449358b3277530a7eff6c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:46.552839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0f6dc-0048-42d5-a106-a7a3029d6ff1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-9a6b0ce1-1a8f-43fc-a35d-e26565c5d0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:46.561420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:46.562624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:46.563699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:46.563699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:46.563699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:46.567633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 312/586] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:46.568655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b0ce1-1a8f-43fc-a35d-e26565c5d0ca None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:46.699277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b0ce1-1a8f-43fc-a35d-e26565c5d0ca None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:46.711049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0f6dc-0048-42d5-a106-a7a3029d6ff1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:46.726951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:46.740875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a4d72-905b-46d1-86c9-9c26642d4a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:46.749647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a4d72-905b-46d1-86c9-9c26642d4a70 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:46.749647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a4d72-905b-46d1-86c9-9c26642d4a70 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:46.749647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a4d72-905b-46d1-86c9-9c26642d4a70 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:46.751256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a4d72-905b-46d1-86c9-9c26642d4a70 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:46.752869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a4d72-905b-46d1-86c9-9c26642d4a70 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:46.756910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ed54c-ba1f-4228-b912-35dcae5a5d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:46.757791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ed54c-ba1f-4228-b912-35dcae5a5d92 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:46.758253 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ed54c-ba1f-4228-b912-35dcae5a5d92 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:46.758706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ed54c-ba1f-4228-b912-35dcae5a5d92 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:46.759554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ed54c-ba1f-4228-b912-35dcae5a5d92 None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:46.760690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ed54c-ba1f-4228-b912-35dcae5a5d92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:46.762256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ed54c-ba1f-4228-b912-35dcae5a5d92 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-535a4d72-905b-46d1-86c9-9c26642d4a70 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:46.767585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:46.768614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 313/587] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 11:40:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:46.769550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a4d72-905b-46d1-86c9-9c26642d4a70 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:46.782761 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 275/588] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:46.785950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ed54c-ba1f-4228-b912-35dcae5a5d92 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:46.792371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:46.899251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a4d72-905b-46d1-86c9-9c26642d4a70 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:46.901563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ed54c-ba1f-4228-b912-35dcae5a5d92 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:46.941848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326e4b1-0a0e-4e05-8d37-7079ad947364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:46.942321 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708c1f8-8b8e-4f93-929b-69d9aa174e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:46.942754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326e4b1-0a0e-4e05-8d37-7079ad947364 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:46.943162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326e4b1-0a0e-4e05-8d37-7079ad947364 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:46.943485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708c1f8-8b8e-4f93-929b-69d9aa174e39 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:46.943686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326e4b1-0a0e-4e05-8d37-7079ad947364 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:46.944071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708c1f8-8b8e-4f93-929b-69d9aa174e39 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:46.944539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708c1f8-8b8e-4f93-929b-69d9aa174e39 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:46.944694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326e4b1-0a0e-4e05-8d37-7079ad947364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:46.945105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326e4b1-0a0e-4e05-8d37-7079ad947364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:46.945717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708c1f8-8b8e-4f93-929b-69d9aa174e39 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:46.946175 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708c1f8-8b8e-4f93-929b-69d9aa174e39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:46.946834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326e4b1-0a0e-4e05-8d37-7079ad947364 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:46.948582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708c1f8-8b8e-4f93-929b-69d9aa174e39 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:46.962486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 314/589] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:46.963712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326e4b1-0a0e-4e05-8d37-7079ad947364 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:46.983777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 276/590] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 11:40:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:46.984756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708c1f8-8b8e-4f93-929b-69d9aa174e39 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:47.024480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:47.074169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:47.117793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326e4b1-0a0e-4e05-8d37-7079ad947364 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:47.147896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708c1f8-8b8e-4f93-929b-69d9aa174e39 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:47.162294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319af6a2-1503-4880-a1ea-76943081de00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:47.163281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319af6a2-1503-4880-a1ea-76943081de00 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.163740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319af6a2-1503-4880-a1ea-76943081de00 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.164161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319af6a2-1503-4880-a1ea-76943081de00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.176292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319af6a2-1503-4880-a1ea-76943081de00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:47.178037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319af6a2-1503-4880-a1ea-76943081de00 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-319af6a2-1503-4880-a1ea-76943081de00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:47.196582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:47.197869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:47.199010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:47.199010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:47.199010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:47.200387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faddc6-4f4d-428b-858f-cb0650b50081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:47.201198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faddc6-4f4d-428b-858f-cb0650b50081 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.201358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 315/591] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:47.201643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faddc6-4f4d-428b-858f-cb0650b50081 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.202070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faddc6-4f4d-428b-858f-cb0650b50081 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.202782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319af6a2-1503-4880-a1ea-76943081de00 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:47.211712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6faddc6-4f4d-428b-858f-cb0650b50081 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:47.214732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6faddc6-4f4d-428b-858f-cb0650b50081 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-a6faddc6-4f4d-428b-858f-cb0650b50081 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:47.233355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:47.234486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:47.235604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:47.235604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:47.235604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:47.235604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:47.235604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:47.235604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:47.235604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:47.235604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:47.240263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 277/592] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:47.241380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faddc6-4f4d-428b-858f-cb0650b50081 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:47.345151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:47.354339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319af6a2-1503-4880-a1ea-76943081de00 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:47.360164 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faddc6-4f4d-428b-858f-cb0650b50081 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:47.388893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:47.397750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e032ed-44c4-4e9c-b4f9-eb9084293333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:47.398473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e032ed-44c4-4e9c-b4f9-eb9084293333 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.398830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e032ed-44c4-4e9c-b4f9-eb9084293333 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.399382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e032ed-44c4-4e9c-b4f9-eb9084293333 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.399898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e032ed-44c4-4e9c-b4f9-eb9084293333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:47.400309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e032ed-44c4-4e9c-b4f9-eb9084293333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:47.402034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e032ed-44c4-4e9c-b4f9-eb9084293333 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:47.405799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb7dce-2ae4-480e-aeb8-5f6fc67dd646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:47.406661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb7dce-2ae4-480e-aeb8-5f6fc67dd646 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.407112 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb7dce-2ae4-480e-aeb8-5f6fc67dd646 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.408307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb7dce-2ae4-480e-aeb8-5f6fc67dd646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.415895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 278/593] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:47.416993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e032ed-44c4-4e9c-b4f9-eb9084293333 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:47.424045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb7dce-2ae4-480e-aeb8-5f6fc67dd646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:47.426254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb7dce-2ae4-480e-aeb8-5f6fc67dd646 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-bbdb7dce-2ae4-480e-aeb8-5f6fc67dd646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:47.449693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:47.450736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:47.451647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:47.451647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:47.451647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:47.451647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 316/594] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:47.452442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb7dce-2ae4-480e-aeb8-5f6fc67dd646 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:47.564462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e032ed-44c4-4e9c-b4f9-eb9084293333 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:47.569965 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb7dce-2ae4-480e-aeb8-5f6fc67dd646 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:47.606930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b501724-30a8-4690-bccc-27476e93704c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:47.607827 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b501724-30a8-4690-bccc-27476e93704c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.608211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b501724-30a8-4690-bccc-27476e93704c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.608598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b501724-30a8-4690-bccc-27476e93704c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.608854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcdd5c-678c-41e3-ba86-6c2b5d5426dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:47.609562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcdd5c-678c-41e3-ba86-6c2b5d5426dc None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.609910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcdd5c-678c-41e3-ba86-6c2b5d5426dc None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.610253 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcdd5c-678c-41e3-ba86-6c2b5d5426dc None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.611013 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcdd5c-678c-41e3-ba86-6c2b5d5426dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:47.611393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcdd5c-678c-41e3-ba86-6c2b5d5426dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:47.612929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcdd5c-678c-41e3-ba86-6c2b5d5426dc None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:47.617080 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b501724-30a8-4690-bccc-27476e93704c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:47.618597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b501724-30a8-4690-bccc-27476e93704c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:47.626166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 279/595] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:47.627145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcdd5c-678c-41e3-ba86-6c2b5d5426dc None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-6b501724-30a8-4690-bccc-27476e93704c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:47.635904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:47.637096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:47.638255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:47.638255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:47.638255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:47.638255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:47.638255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:47.638255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:47.638255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:47.638255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:47.638255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 11:40:47.638255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:47.640817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 317/596] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:47.640817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5191570-f84b-4031-a775-9147a7329e25 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.640817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5191570-f84b-4031-a775-9147a7329e25 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.641039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5191570-f84b-4031-a775-9147a7329e25 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.642477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 318/597] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:47.643325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5191570-f84b-4031-a775-9147a7329e25 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:47.643677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:47.685456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:47.745840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcdd5c-678c-41e3-ba86-6c2b5d5426dc None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:47.761062 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5191570-f84b-4031-a775-9147a7329e25 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:47.781366 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b5f3e-9787-41db-b115-ecc50f8a8627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:47.785148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5f3e-9787-41db-b115-ecc50f8a8627 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.785148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5f3e-9787-41db-b115-ecc50f8a8627 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.785148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5f3e-9787-41db-b115-ecc50f8a8627 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.785148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b5f3e-9787-41db-b115-ecc50f8a8627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:47.785148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b5f3e-9787-41db-b115-ecc50f8a8627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:47.785148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b5f3e-9787-41db-b115-ecc50f8a8627 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:47.801980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 280/598] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:47.802915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b5f3e-9787-41db-b115-ecc50f8a8627 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:47.811584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a02735-7f51-4c1b-a0dd-22e4c9ce76ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:47.812291 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a02735-7f51-4c1b-a0dd-22e4c9ce76ee None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.812725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a02735-7f51-4c1b-a0dd-22e4c9ce76ee None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.813000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a02735-7f51-4c1b-a0dd-22e4c9ce76ee None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/496a2e8bf591437888ea82161ca4d5ee/roles/5b919ba5900449358b3277530a7eff6c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.829384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a02735-7f51-4c1b-a0dd-22e4c9ce76ee None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=496a2e8bf591437888ea82161ca4d5ee, role_id=5b919ba5900449358b3277530a7eff6c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:47.830833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a02735-7f51-4c1b-a0dd-22e4c9ce76ee None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:47.873115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:47.906436 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b5f3e-9787-41db-b115-ecc50f8a8627 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:47.920201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 319/599] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:40:47 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/496a2e8bf591437888ea82161ca4d5ee/roles/5b919ba5900449358b3277530a7eff6c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:47.920201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62881a21-5e0a-42e8-930a-de74e573f664 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.920201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62881a21-5e0a-42e8-930a-de74e573f664 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.920201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62881a21-5e0a-42e8-930a-de74e573f664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.941933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-62881a21-5e0a-42e8-930a-de74e573f664 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:47.943372 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-62881a21-5e0a-42e8-930a-de74e573f664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:47.948098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be735bac-c0b7-46e8-8296-56b6e1321678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:47.948768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be735bac-c0b7-46e8-8296-56b6e1321678 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:47.949049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be735bac-c0b7-46e8-8296-56b6e1321678 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:47.949324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be735bac-c0b7-46e8-8296-56b6e1321678 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:47.950079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be735bac-c0b7-46e8-8296-56b6e1321678 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:47.950390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be735bac-c0b7-46e8-8296-56b6e1321678 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:47.951901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be735bac-c0b7-46e8-8296-56b6e1321678 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:47.979581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 281/600] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Oct 4 11:40:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:47.979581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be735bac-c0b7-46e8-8296-56b6e1321678 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:48.051494 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 320/601] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:48.051494 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62881a21-5e0a-42e8-930a-de74e573f664 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:48.069039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:48.144721 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be735bac-c0b7-46e8-8296-56b6e1321678 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:48.172805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62881a21-5e0a-42e8-930a-de74e573f664 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:48.179208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:48.186949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2a5d2-a8f2-4d3e-9e0b-53b388ece9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:48.187964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2a5d2-a8f2-4d3e-9e0b-53b388ece9d8 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:48.188475 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2a5d2-a8f2-4d3e-9e0b-53b388ece9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:48.189105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2a5d2-a8f2-4d3e-9e0b-53b388ece9d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:48.203485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2a5d2-a8f2-4d3e-9e0b-53b388ece9d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:48.208067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2a5d2-a8f2-4d3e-9e0b-53b388ece9d8 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-70e2a5d2-a8f2-4d3e-9e0b-53b388ece9d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:48.225786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:48.226758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:48.227740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:48.227740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:48.227740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:48.227740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:48.227740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:48.227740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:48.227740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:48.227740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:48.230406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 282/602] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:48.236450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2a5d2-a8f2-4d3e-9e0b-53b388ece9d8 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:48.261714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10a394-fba8-4434-b832-5f46e7e02fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:48.262366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a394-fba8-4434-b832-5f46e7e02fbf None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:48.262645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a394-fba8-4434-b832-5f46e7e02fbf None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:48.262913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10a394-fba8-4434-b832-5f46e7e02fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:48.301052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10a394-fba8-4434-b832-5f46e7e02fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:48.302525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10a394-fba8-4434-b832-5f46e7e02fbf None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-de10a394-fba8-4434-b832-5f46e7e02fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:48.343145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:48.344161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:48.345107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:48.345107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:48.345107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:48.346475 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 321/603] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:48.351355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10a394-fba8-4434-b832-5f46e7e02fbf None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:48.371323 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2a5d2-a8f2-4d3e-9e0b-53b388ece9d8 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:48.417039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d7720-ed47-4379-af7f-dcca9766116e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:48.417563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d7720-ed47-4379-af7f-dcca9766116e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:48.418211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d7720-ed47-4379-af7f-dcca9766116e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:48.418513 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d7720-ed47-4379-af7f-dcca9766116e None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:48.419202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d7720-ed47-4379-af7f-dcca9766116e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:48.419507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d7720-ed47-4379-af7f-dcca9766116e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:48.421187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d7720-ed47-4379-af7f-dcca9766116e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:48.435675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:48.438820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 283/604] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:48.439895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d7720-ed47-4379-af7f-dcca9766116e None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:48.474560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10a394-fba8-4434-b832-5f46e7e02fbf None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:48.501789 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:48.521641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765752d-2b45-427a-a28d-76a03c9cc3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:48.522457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765752d-2b45-427a-a28d-76a03c9cc3e3 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:48.522747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765752d-2b45-427a-a28d-76a03c9cc3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:48.523061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765752d-2b45-427a-a28d-76a03c9cc3e3 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:48.523711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765752d-2b45-427a-a28d-76a03c9cc3e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:48.524013 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765752d-2b45-427a-a28d-76a03c9cc3e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:48.526386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765752d-2b45-427a-a28d-76a03c9cc3e3 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:48.539354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 322/605] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:48.540281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765752d-2b45-427a-a28d-76a03c9cc3e3 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:48.563922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d7720-ed47-4379-af7f-dcca9766116e None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:48.610744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edf87f-cb87-47be-8b95-0b296700b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:48.611492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edf87f-cb87-47be-8b95-0b296700b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:48.611833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edf87f-cb87-47be-8b95-0b296700b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:48.612235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edf87f-cb87-47be-8b95-0b296700b3e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:48.624094 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edf87f-cb87-47be-8b95-0b296700b3e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-c3edf87f-cb87-47be-8b95-0b296700b3e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:48.629750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:48.630730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:48.631578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 284/606] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:48.632320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edf87f-cb87-47be-8b95-0b296700b3e0 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:48.657897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765752d-2b45-427a-a28d-76a03c9cc3e3 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:48.689012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:48.702055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50046776-2108-4cb4-b93f-1e7b662e6a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:48.702870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046776-2108-4cb4-b93f-1e7b662e6a47 None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:48.703402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046776-2108-4cb4-b93f-1e7b662e6a47 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:48.703689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046776-2108-4cb4-b93f-1e7b662e6a47 None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/ed14c1bc105d4c5cade5813edc1f0dac/roles/5b919ba5900449358b3277530a7eff6c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:48.723471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50046776-2108-4cb4-b93f-1e7b662e6a47 None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=ed14c1bc105d4c5cade5813edc1f0dac, role_id=5b919ba5900449358b3277530a7eff6c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:48.725533 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50046776-2108-4cb4-b93f-1e7b662e6a47 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:48.752272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edf87f-cb87-47be-8b95-0b296700b3e0 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:48.761795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:48.801289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a79899-98ce-47c6-b270-bed531298eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:48.802227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a79899-98ce-47c6-b270-bed531298eb4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:48.802509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a79899-98ce-47c6-b270-bed531298eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:48.806423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a79899-98ce-47c6-b270-bed531298eb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:48.811379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a79899-98ce-47c6-b270-bed531298eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:48.812873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a79899-98ce-47c6-b270-bed531298eb4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-29a79899-98ce-47c6-b270-bed531298eb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:48.825381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:48.826425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:48.827988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:48.827988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:48.827988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:48.827988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:48.827988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:48.827988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:48.827988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:48.827988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:48.831178 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 285/607] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:48.831979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a79899-98ce-47c6-b270-bed531298eb4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:48.891619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 323/608] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:40:48 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/ed14c1bc105d4c5cade5813edc1f0dac/roles/5b919ba5900449358b3277530a7eff6c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:48.891619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50046776-2108-4cb4-b93f-1e7b662e6a47 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:48.973903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a79899-98ce-47c6-b270-bed531298eb4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:49.022676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50046776-2108-4cb4-b93f-1e7b662e6a47 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:49.023652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6b527-c66e-4c39-8124-f3215de7b52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:49.024595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6b527-c66e-4c39-8124-f3215de7b52b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:49.025022 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6b527-c66e-4c39-8124-f3215de7b52b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:49.026343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6b527-c66e-4c39-8124-f3215de7b52b None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:49.027153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6b527-c66e-4c39-8124-f3215de7b52b None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:49.027564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6b527-c66e-4c39-8124-f3215de7b52b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:49.029127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6b527-c66e-4c39-8124-f3215de7b52b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:49.041674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 286/609] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:49.043516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c9e7b-99f5-4b33-bc6c-575dbf2b3bf1 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:49.043801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c9e7b-99f5-4b33-bc6c-575dbf2b3bf1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:49.044077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c9e7b-99f5-4b33-bc6c-575dbf2b3bf1 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:49.048286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 287/610] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:49.049391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c9e7b-99f5-4b33-bc6c-575dbf2b3bf1 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:49.069933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c293835-0b40-4866-adee-8cde64d48415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:49.070873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c293835-0b40-4866-adee-8cde64d48415 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:49.071343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c293835-0b40-4866-adee-8cde64d48415 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:49.071720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c293835-0b40-4866-adee-8cde64d48415 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:49.072640 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c293835-0b40-4866-adee-8cde64d48415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:49.073019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c293835-0b40-4866-adee-8cde64d48415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:49.074783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c293835-0b40-4866-adee-8cde64d48415 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:49.088160 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:49.094781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 324/611] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:49.096201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad67c70-6e1c-4984-8311-055a783aa43f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:49.096201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad67c70-6e1c-4984-8311-055a783aa43f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:49.096201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad67c70-6e1c-4984-8311-055a783aa43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:49.118419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4ad67c70-6e1c-4984-8311-055a783aa43f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:49.120286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4ad67c70-6e1c-4984-8311-055a783aa43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:49.164888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:49.192348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c9e7b-99f5-4b33-bc6c-575dbf2b3bf1 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:49.214072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 325/612] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:49.219597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad67c70-6e1c-4984-8311-055a783aa43f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:49.250780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53183c6a-a0f8-4828-a43c-766f293ccacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:49.253580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53183c6a-a0f8-4828-a43c-766f293ccacf None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:49.254139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53183c6a-a0f8-4828-a43c-766f293ccacf None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:49.254641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53183c6a-a0f8-4828-a43c-766f293ccacf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:49.282292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53183c6a-a0f8-4828-a43c-766f293ccacf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:49.285626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53183c6a-a0f8-4828-a43c-766f293ccacf None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-53183c6a-a0f8-4828-a43c-766f293ccacf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:49.316403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:49.317419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:49.318322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:49.318322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:49.318322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:49.325287 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 288/613] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:49.328294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53183c6a-a0f8-4828-a43c-766f293ccacf None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:49.433571 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad67c70-6e1c-4984-8311-055a783aa43f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:49.491477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:49.501526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa915ba9-d0fe-4365-b5cd-b354f7fe291c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:49.502410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa915ba9-d0fe-4365-b5cd-b354f7fe291c None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:49.502839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa915ba9-d0fe-4365-b5cd-b354f7fe291c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:49.503317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa915ba9-d0fe-4365-b5cd-b354f7fe291c None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/073e43c049224af29e9f6d426ea47e84/roles/5b919ba5900449358b3277530a7eff6c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:49.515609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53183c6a-a0f8-4828-a43c-766f293ccacf None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:49.522178 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa915ba9-d0fe-4365-b5cd-b354f7fe291c None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=073e43c049224af29e9f6d426ea47e84, role_id=5b919ba5900449358b3277530a7eff6c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:49.523945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa915ba9-d0fe-4365-b5cd-b354f7fe291c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:49.555746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9386-5d4a-4da2-8d82-ee7e981882e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:49.556606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9386-5d4a-4da2-8d82-ee7e981882e4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:49.557507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9386-5d4a-4da2-8d82-ee7e981882e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:49.557890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9386-5d4a-4da2-8d82-ee7e981882e4 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:49.558706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d9386-5d4a-4da2-8d82-ee7e981882e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:49.559156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d9386-5d4a-4da2-8d82-ee7e981882e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:49.560928 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d9386-5d4a-4da2-8d82-ee7e981882e4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:49.574055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 289/614] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:49.574762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9386-5d4a-4da2-8d82-ee7e981882e4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:49.602651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 326/615] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:40:49 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/073e43c049224af29e9f6d426ea47e84/roles/5b919ba5900449358b3277530a7eff6c => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:49.602651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa915ba9-d0fe-4365-b5cd-b354f7fe291c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:49.637123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:49.752087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9386-5d4a-4da2-8d82-ee7e981882e4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:49.814932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa915ba9-d0fe-4365-b5cd-b354f7fe291c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:49.826358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0101cb-7b97-4198-9c04-449c9d10302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:49.827458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0101cb-7b97-4198-9c04-449c9d10302b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:49.827976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0101cb-7b97-4198-9c04-449c9d10302b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:49.828859 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0101cb-7b97-4198-9c04-449c9d10302b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:49.841379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:49.847636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0101cb-7b97-4198-9c04-449c9d10302b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-6f0101cb-7b97-4198-9c04-449c9d10302b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:49.854392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:49.858274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:49.859692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:49.860468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 290/616] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:49.862510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76a8f1-f49b-4ce9-89ab-8d3d8be7c12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:49.863225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0101cb-7b97-4198-9c04-449c9d10302b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:49.863534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76a8f1-f49b-4ce9-89ab-8d3d8be7c12b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:49.864134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76a8f1-f49b-4ce9-89ab-8d3d8be7c12b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:49.864677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76a8f1-f49b-4ce9-89ab-8d3d8be7c12b None admin] PATH_INFO: `/v3/users/swift` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:49.874145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76a8f1-f49b-4ce9-89ab-8d3d8be7c12b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:49.875885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76a8f1-f49b-4ce9-89ab-8d3d8be7c12b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-cc76a8f1-f49b-4ce9-89ab-8d3d8be7c12b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:49.889537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:49.890711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:49.891678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:49.891678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:49.891678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:49.891678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:49.891678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:49.891678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:49.891678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:49.891678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:49.891678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:40:49.891678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:49.894044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 327/617] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:40:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:49.900461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76a8f1-f49b-4ce9-89ab-8d3d8be7c12b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:50.046263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:50.047412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0101cb-7b97-4198-9c04-449c9d10302b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:50.083743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76a8f1-f49b-4ce9-89ab-8d3d8be7c12b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:50.121485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59bf16-43bf-425c-9b1b-4f2dded21157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:50.122202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59bf16-43bf-425c-9b1b-4f2dded21157 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:50.122494 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59bf16-43bf-425c-9b1b-4f2dded21157 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:50.122842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59bf16-43bf-425c-9b1b-4f2dded21157 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:50.123412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77075642-dbe2-4213-9192-f98b693ab7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:50.123622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59bf16-43bf-425c-9b1b-4f2dded21157 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:50.123899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59bf16-43bf-425c-9b1b-4f2dded21157 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:50.124626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075642-dbe2-4213-9192-f98b693ab7e1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:50.125090 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075642-dbe2-4213-9192-f98b693ab7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:50.125388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59bf16-43bf-425c-9b1b-4f2dded21157 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:50.125792 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075642-dbe2-4213-9192-f98b693ab7e1 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:50.126648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77075642-dbe2-4213-9192-f98b693ab7e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:50.127158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77075642-dbe2-4213-9192-f98b693ab7e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:50.128875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77075642-dbe2-4213-9192-f98b693ab7e1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:50.148824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 291/619] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:50.149095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 328/619] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 11:40:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:50.153572 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59bf16-43bf-425c-9b1b-4f2dded21157 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:50.154842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77075642-dbe2-4213-9192-f98b693ab7e1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:50.301416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:50.357564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59bf16-43bf-425c-9b1b-4f2dded21157 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:50.390234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77075642-dbe2-4213-9192-f98b693ab7e1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:50.428002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8630b-2be3-4479-822d-3651e70dd6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:50.429008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8630b-2be3-4479-822d-3651e70dd6cd None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:50.429288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8630b-2be3-4479-822d-3651e70dd6cd None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:50.429566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8630b-2be3-4479-822d-3651e70dd6cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:50.434578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:50.453424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8630b-2be3-4479-822d-3651e70dd6cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:50.454899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8630b-2be3-4479-822d-3651e70dd6cd None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:50.461326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a115b-f03c-4127-a9f1-fdd5b05e136e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:50.462226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a115b-f03c-4127-a9f1-fdd5b05e136e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:50.462712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a115b-f03c-4127-a9f1-fdd5b05e136e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:50.463337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a115b-f03c-4127-a9f1-fdd5b05e136e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-a7c8630b-2be3-4479-822d-3651e70dd6cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:50.472630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:50.473632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:50.474535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:50.474535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:50.474535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:50.474535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 329/620] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:50.475194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14ff0a-e713-4bc4-9f0c-a6dc8bc812c2 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:50.475528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14ff0a-e713-4bc4-9f0c-a6dc8bc812c2 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:50.476133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14ff0a-e713-4bc4-9f0c-a6dc8bc812c2 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:50.480726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a115b-f03c-4127-a9f1-fdd5b05e136e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:50.480957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 330/621] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:50.481900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14ff0a-e713-4bc4-9f0c-a6dc8bc812c2 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:50.483045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a115b-f03c-4127-a9f1-fdd5b05e136e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-dd0a115b-f03c-4127-a9f1-fdd5b05e136e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:50.501481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:50.503156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:50.504365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:50.504365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:50.504365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:50.506238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 292/622] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:50.507730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a4e2b-4397-44e4-8012-0683d59ddd00 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:50.508185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a4e2b-4397-44e4-8012-0683d59ddd00 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:50.508661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a4e2b-4397-44e4-8012-0683d59ddd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:50.529487 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-eb7a4e2b-4397-44e4-8012-0683d59ddd00 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:50.530943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-eb7a4e2b-4397-44e4-8012-0683d59ddd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:50.598603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14ff0a-e713-4bc4-9f0c-a6dc8bc812c2 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:50.607739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 293/623] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:50.608884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a4e2b-4397-44e4-8012-0683d59ddd00 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:50.641582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91737b9-12ad-4047-8976-2b4913f2ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:50.643729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91737b9-12ad-4047-8976-2b4913f2ac88 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:50.644334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91737b9-12ad-4047-8976-2b4913f2ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:50.645392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91737b9-12ad-4047-8976-2b4913f2ac88 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:50.646159 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91737b9-12ad-4047-8976-2b4913f2ac88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:50.646795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91737b9-12ad-4047-8976-2b4913f2ac88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:50.651192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91737b9-12ad-4047-8976-2b4913f2ac88 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:50.684981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 331/624] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:50.692196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91737b9-12ad-4047-8976-2b4913f2ac88 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:50.710583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:50.787632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a4e2b-4397-44e4-8012-0683d59ddd00 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:50.799867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:50.827799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7ce1c-e725-4850-92b4-2dd724540d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:50.828826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7ce1c-e725-4850-92b4-2dd724540d82 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:50.829257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7ce1c-e725-4850-92b4-2dd724540d82 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:50.829681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7ce1c-e725-4850-92b4-2dd724540d82 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:50.830496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7ce1c-e725-4850-92b4-2dd724540d82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:50.830943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7ce1c-e725-4850-92b4-2dd724540d82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:50.832493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7ce1c-e725-4850-92b4-2dd724540d82 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:50.847044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 294/625] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:50.852854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7ce1c-e725-4850-92b4-2dd724540d82 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:50.929853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91737b9-12ad-4047-8976-2b4913f2ac88 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:50.975137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cbe30-06e6-4b87-bc8a-e90e41af873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:50.975771 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cbe30-06e6-4b87-bc8a-e90e41af873d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:50.976052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cbe30-06e6-4b87-bc8a-e90e41af873d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:50.976361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cbe30-06e6-4b87-bc8a-e90e41af873d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:50.977023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7ce1c-e725-4850-92b4-2dd724540d82 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:50.979856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:50.985323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cbe30-06e6-4b87-bc8a-e90e41af873d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:50.987002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cbe30-06e6-4b87-bc8a-e90e41af873d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-858cbe30-06e6-4b87-bc8a-e90e41af873d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:51.002257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:51.003240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:51.004155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:51.004155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:51.004155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:51.004155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:51.004155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:51.004155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:51.004155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:51.004155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:51.006588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 332/626] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:51.008576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cbe30-06e6-4b87-bc8a-e90e41af873d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:51.018717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6ac71-cff2-4eea-b7f0-c1ad59c99d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:51.019459 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6ac71-cff2-4eea-b7f0-c1ad59c99d86 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:51.019744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6ac71-cff2-4eea-b7f0-c1ad59c99d86 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:51.020023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6ac71-cff2-4eea-b7f0-c1ad59c99d86 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:51.029447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6ac71-cff2-4eea-b7f0-c1ad59c99d86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:51.030969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6ac71-cff2-4eea-b7f0-c1ad59c99d86 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-ddc6ac71-cff2-4eea-b7f0-c1ad59c99d86 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:51.043852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:51.044886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:51.045856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 295/627] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:51.046452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6ac71-cff2-4eea-b7f0-c1ad59c99d86 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:51.146346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cbe30-06e6-4b87-bc8a-e90e41af873d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:51.153241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:51.190891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d947ca-d1ef-4b54-869f-0106c39b4a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:51.192469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d947ca-d1ef-4b54-869f-0106c39b4a8e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:51.192794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d947ca-d1ef-4b54-869f-0106c39b4a8e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:51.194013 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d947ca-d1ef-4b54-869f-0106c39b4a8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:51.207578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d947ca-d1ef-4b54-869f-0106c39b4a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:51.210733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6ac71-cff2-4eea-b7f0-c1ad59c99d86 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-f3d947ca-d1ef-4b54-869f-0106c39b4a8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:51.213409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:51.215361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 333/628] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:51.217707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d947ca-d1ef-4b54-869f-0106c39b4a8e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:51.284645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89b4c-1f5a-4cf4-bf64-c0ad84ca446d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:51.285984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89b4c-1f5a-4cf4-bf64-c0ad84ca446d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:51.286765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89b4c-1f5a-4cf4-bf64-c0ad84ca446d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:51.287652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89b4c-1f5a-4cf4-bf64-c0ad84ca446d None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:51.289360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d89b4c-1f5a-4cf4-bf64-c0ad84ca446d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:51.290156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d89b4c-1f5a-4cf4-bf64-c0ad84ca446d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:51.294011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d89b4c-1f5a-4cf4-bf64-c0ad84ca446d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:51.317122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 296/629] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:51.319163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89b4c-1f5a-4cf4-bf64-c0ad84ca446d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:51.328858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d947ca-d1ef-4b54-869f-0106c39b4a8e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:51.365073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:51.366028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a16159-ea7e-4630-9ea3-f688fa09d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:51.366638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a16159-ea7e-4630-9ea3-f688fa09d59e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:51.366918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a16159-ea7e-4630-9ea3-f688fa09d59e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:51.367191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a16159-ea7e-4630-9ea3-f688fa09d59e None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:51.367913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a16159-ea7e-4630-9ea3-f688fa09d59e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:51.368243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a16159-ea7e-4630-9ea3-f688fa09d59e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:51.370455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a16159-ea7e-4630-9ea3-f688fa09d59e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:51.389992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 334/630] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 11:40:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:51.391646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f02cf-9a19-4be4-8553-a129e12ba080 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:51.392141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f02cf-9a19-4be4-8553-a129e12ba080 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:51.392652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f02cf-9a19-4be4-8553-a129e12ba080 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:51.398118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 335/631] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:51.398118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f02cf-9a19-4be4-8553-a129e12ba080 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:51.423528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:51.458050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89b4c-1f5a-4cf4-bf64-c0ad84ca446d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:51.497255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88c391-d924-4f4e-b5c3-67c5933b9dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:51.497855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88c391-d924-4f4e-b5c3-67c5933b9dbd None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:51.498126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88c391-d924-4f4e-b5c3-67c5933b9dbd None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:51.498397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88c391-d924-4f4e-b5c3-67c5933b9dbd None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:51.499083 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88c391-d924-4f4e-b5c3-67c5933b9dbd None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:51.499382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88c391-d924-4f4e-b5c3-67c5933b9dbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:51.501095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88c391-d924-4f4e-b5c3-67c5933b9dbd None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:51.510873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 297/632] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:51.511749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88c391-d924-4f4e-b5c3-67c5933b9dbd None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:51.522036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f02cf-9a19-4be4-8553-a129e12ba080 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:51.561417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1543d0f-8e31-48e3-9ad6-2faf247ac07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:51.562332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1543d0f-8e31-48e3-9ad6-2faf247ac07a None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:51.562769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1543d0f-8e31-48e3-9ad6-2faf247ac07a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:51.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1543d0f-8e31-48e3-9ad6-2faf247ac07a None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/734d8f03ff2949959cbfce98212a3ac6/roles/5b919ba5900449358b3277530a7eff6c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:51.581610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1543d0f-8e31-48e3-9ad6-2faf247ac07a None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=734d8f03ff2949959cbfce98212a3ac6, role_id=5b919ba5900449358b3277530a7eff6c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:51.583033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1543d0f-8e31-48e3-9ad6-2faf247ac07a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:51.610027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:51.625167 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88c391-d924-4f4e-b5c3-67c5933b9dbd None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:51.653628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 336/633] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:40:51 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/734d8f03ff2949959cbfce98212a3ac6/roles/5b919ba5900449358b3277530a7eff6c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:40:51.655312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1543b7f-3487-4205-aa43-be85fbcfec07 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:51.655784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1543b7f-3487-4205-aa43-be85fbcfec07 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:51.656566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1543b7f-3487-4205-aa43-be85fbcfec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:51.672213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f294f3d-aeb9-47fd-b2bb-13b1c3a44b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:51.673020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f294f3d-aeb9-47fd-b2bb-13b1c3a44b4c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:51.673309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f294f3d-aeb9-47fd-b2bb-13b1c3a44b4c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:51.673582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f294f3d-aeb9-47fd-b2bb-13b1c3a44b4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:51.684851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b1543b7f-3487-4205-aa43-be85fbcfec07 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:51.687357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b1543b7f-3487-4205-aa43-be85fbcfec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:51.773934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f294f3d-aeb9-47fd-b2bb-13b1c3a44b4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:51.775563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f294f3d-aeb9-47fd-b2bb-13b1c3a44b4c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:51.824210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 337/634] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:51.825875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1543b7f-3487-4205-aa43-be85fbcfec07 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:51.832937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:51.934495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:51.977426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 298/635] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:51.979663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f294f3d-aeb9-47fd-b2bb-13b1c3a44b4c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:52.012795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1543b7f-3487-4205-aa43-be85fbcfec07 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:52.066457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58026b9-5806-46a5-9f19-c7095b61789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:52.067193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58026b9-5806-46a5-9f19-c7095b61789e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.067479 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58026b9-5806-46a5-9f19-c7095b61789e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.067748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58026b9-5806-46a5-9f19-c7095b61789e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.079419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58026b9-5806-46a5-9f19-c7095b61789e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:52.080911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58026b9-5806-46a5-9f19-c7095b61789e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-e58026b9-5806-46a5-9f19-c7095b61789e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:52.097620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:52.098632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:52.099519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:52.099519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:52.099519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:52.099519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 338/636] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:52.109433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58026b9-5806-46a5-9f19-c7095b61789e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:52.132844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f294f3d-aeb9-47fd-b2bb-13b1c3a44b4c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:52.188688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:52.195209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7a1e1-4bd7-4b2e-84d5-18149e00f755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:52.195898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7a1e1-4bd7-4b2e-84d5-18149e00f755 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.196374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7a1e1-4bd7-4b2e-84d5-18149e00f755 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.196699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7a1e1-4bd7-4b2e-84d5-18149e00f755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.209208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7a1e1-4bd7-4b2e-84d5-18149e00f755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:52.210967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7a1e1-4bd7-4b2e-84d5-18149e00f755 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-9ae7a1e1-4bd7-4b2e-84d5-18149e00f755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:52.227039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:52.230998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:52.232702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:52.232702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:52.232702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:52.232702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 299/637] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:52.232702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7a1e1-4bd7-4b2e-84d5-18149e00f755 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:52.241559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58026b9-5806-46a5-9f19-c7095b61789e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:52.276444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:52.281552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7c7ad-765a-4d3c-82b2-3e04e4a88372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:52.282240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7c7ad-765a-4d3c-82b2-3e04e4a88372 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.282522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7c7ad-765a-4d3c-82b2-3e04e4a88372 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.282801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7c7ad-765a-4d3c-82b2-3e04e4a88372 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.283494 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7c7ad-765a-4d3c-82b2-3e04e4a88372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:52.283804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7c7ad-765a-4d3c-82b2-3e04e4a88372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:52.285330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7c7ad-765a-4d3c-82b2-3e04e4a88372 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:52.300606 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 339/638] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:52.301515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7c7ad-765a-4d3c-82b2-3e04e4a88372 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:52.350725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7a1e1-4bd7-4b2e-84d5-18149e00f755 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:52.388812 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b8715-e620-4315-8b62-872d58998d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:52.389541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b8715-e620-4315-8b62-872d58998d1c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.389896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b8715-e620-4315-8b62-872d58998d1c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.390234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b8715-e620-4315-8b62-872d58998d1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.402600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b8715-e620-4315-8b62-872d58998d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-c97b8715-e620-4315-8b62-872d58998d1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:52.408505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:52.409473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:52.410337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:52.414837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 300/639] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:52.415821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b8715-e620-4315-8b62-872d58998d1c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:52.417424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7c7ad-765a-4d3c-82b2-3e04e4a88372 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:52.436525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:52.457219 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9399e46-183c-43ab-8c10-8044f1779528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:52.457752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9399e46-183c-43ab-8c10-8044f1779528 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.458034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9399e46-183c-43ab-8c10-8044f1779528 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.458309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9399e46-183c-43ab-8c10-8044f1779528 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.458952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9399e46-183c-43ab-8c10-8044f1779528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:52.459263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9399e46-183c-43ab-8c10-8044f1779528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:52.460686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9399e46-183c-43ab-8c10-8044f1779528 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:52.473822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 340/640] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:52.479448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9399e46-183c-43ab-8c10-8044f1779528 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:52.532655 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:52.537131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b8715-e620-4315-8b62-872d58998d1c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:52.576929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4fdc7-a3dc-4128-9d60-eefd033cbf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:52.577556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4fdc7-a3dc-4128-9d60-eefd033cbf00 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.577841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4fdc7-a3dc-4128-9d60-eefd033cbf00 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.578119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4fdc7-a3dc-4128-9d60-eefd033cbf00 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.588430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4fdc7-a3dc-4128-9d60-eefd033cbf00 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:52.589853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4fdc7-a3dc-4128-9d60-eefd033cbf00 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:52.602317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9399e46-183c-43ab-8c10-8044f1779528 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-04d4fdc7-a3dc-4128-9d60-eefd033cbf00 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:52.602540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:52.603564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:52.605078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 301/641] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:40:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:52.606205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f49b2-ad42-47f6-9b8b-df59c3d9f6ea None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.606476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f49b2-ad42-47f6-9b8b-df59c3d9f6ea None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.606703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f49b2-ad42-47f6-9b8b-df59c3d9f6ea None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.608747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 302/642] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:52.610352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f49b2-ad42-47f6-9b8b-df59c3d9f6ea None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:52.648581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb3e19-2a0a-42f6-bc4e-1cf6335c458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:52.648796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb3e19-2a0a-42f6-bc4e-1cf6335c458c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.649050 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb3e19-2a0a-42f6-bc4e-1cf6335c458c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.649280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb3e19-2a0a-42f6-bc4e-1cf6335c458c None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.649929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb3e19-2a0a-42f6-bc4e-1cf6335c458c None admin] RBAC: Adding query filter params (name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:52.650238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb3e19-2a0a-42f6-bc4e-1cf6335c458c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:52.652422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb3e19-2a0a-42f6-bc4e-1cf6335c458c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:52.668388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 341/643] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:52.669253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb3e19-2a0a-42f6-bc4e-1cf6335c458c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:52.724240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:52.750289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f49b2-ad42-47f6-9b8b-df59c3d9f6ea None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:52.797696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b44ea-8552-4a90-a578-61bfaf3d3f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:52.799268 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b44ea-8552-4a90-a578-61bfaf3d3f11 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.799712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b44ea-8552-4a90-a578-61bfaf3d3f11 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.801066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b44ea-8552-4a90-a578-61bfaf3d3f11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.809279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b44ea-8552-4a90-a578-61bfaf3d3f11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:52.810791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b44ea-8552-4a90-a578-61bfaf3d3f11 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:52.812938 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb3e19-2a0a-42f6-bc4e-1cf6335c458c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-694b44ea-8552-4a90-a578-61bfaf3d3f11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:52.823909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:52.825159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:52.827256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:52.827256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:52.827256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:52.827256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:52.827256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:52.827256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:52.827256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:52.827256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:52.827256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 303/644] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:52.827256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfe8a0-4741-453b-bbac-7d1ff89b75da None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.827949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfe8a0-4741-453b-bbac-7d1ff89b75da None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.827949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfe8a0-4741-453b-bbac-7d1ff89b75da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.848252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-95dfe8a0-4741-453b-bbac-7d1ff89b75da None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:52.849488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-95dfe8a0-4741-453b-bbac-7d1ff89b75da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:52.863016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43deec-28d6-4f82-9371-00d9e6a29a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:52.863779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43deec-28d6-4f82-9371-00d9e6a29a12 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:52.864164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43deec-28d6-4f82-9371-00d9e6a29a12 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:52.864585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43deec-28d6-4f82-9371-00d9e6a29a12 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:52.865439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43deec-28d6-4f82-9371-00d9e6a29a12 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:52.865843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43deec-28d6-4f82-9371-00d9e6a29a12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:52.867397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43deec-28d6-4f82-9371-00d9e6a29a12 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:52.868265 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:52.889426 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 342/645] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Oct 4 11:40:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:52.890438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43deec-28d6-4f82-9371-00d9e6a29a12 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:52.941944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 304/646] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:52.943010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfe8a0-4741-453b-bbac-7d1ff89b75da None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:53.015077 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:53.021446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43deec-28d6-4f82-9371-00d9e6a29a12 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:53.055012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfe8a0-4741-453b-bbac-7d1ff89b75da None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:53.071438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fca599-fc47-4719-b065-f4d27572d9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:53.072210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fca599-fc47-4719-b065-f4d27572d9ce None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:53.072728 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fca599-fc47-4719-b065-f4d27572d9ce None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:53.073099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fca599-fc47-4719-b065-f4d27572d9ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:53.086018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fca599-fc47-4719-b065-f4d27572d9ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:53.088927 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fca599-fc47-4719-b065-f4d27572d9ce None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:53.095553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9599ef-ab22-4fcd-8989-191ed46a4759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:53.096436 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9599ef-ab22-4fcd-8989-191ed46a4759 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:53.096857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9599ef-ab22-4fcd-8989-191ed46a4759 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:53.097265 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9599ef-ab22-4fcd-8989-191ed46a4759 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:53.098169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9599ef-ab22-4fcd-8989-191ed46a4759 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:53.098626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9599ef-ab22-4fcd-8989-191ed46a4759 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:53.100286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9599ef-ab22-4fcd-8989-191ed46a4759 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-64fca599-fc47-4719-b065-f4d27572d9ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:53.106611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:53.108035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:53.109242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:53.109242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:53.109242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:53.110543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 343/647] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:53.111433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fca599-fc47-4719-b065-f4d27572d9ce None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:53.123413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 305/648] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:53.124643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9599ef-ab22-4fcd-8989-191ed46a4759 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:53.137858 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:53.241635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fca599-fc47-4719-b065-f4d27572d9ce None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:53.250991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9599ef-ab22-4fcd-8989-191ed46a4759 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:53.280661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67144e-680c-4c97-9411-9c4b2edc4862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:53.281214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67144e-680c-4c97-9411-9c4b2edc4862 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:53.281496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67144e-680c-4c97-9411-9c4b2edc4862 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:53.281773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67144e-680c-4c97-9411-9c4b2edc4862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:53.293196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e17d5-c52a-4e99-b5fb-bf525e6d9743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:53.294209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e17d5-c52a-4e99-b5fb-bf525e6d9743 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:53.294754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e17d5-c52a-4e99-b5fb-bf525e6d9743 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:53.295231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e17d5-c52a-4e99-b5fb-bf525e6d9743 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:53.296227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e17d5-c52a-4e99-b5fb-bf525e6d9743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:53.296766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e17d5-c52a-4e99-b5fb-bf525e6d9743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:53.298412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e17d5-c52a-4e99-b5fb-bf525e6d9743 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:53.302563 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:53.313782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 306/649] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:53.314649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e17d5-c52a-4e99-b5fb-bf525e6d9743 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:53.383834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67144e-680c-4c97-9411-9c4b2edc4862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:53.386440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67144e-680c-4c97-9411-9c4b2edc4862 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:53.398086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:53.433482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e17d5-c52a-4e99-b5fb-bf525e6d9743 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:53.469525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bc657-881f-46aa-911e-77548a40bc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:53.470145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bc657-881f-46aa-911e-77548a40bc6e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:53.470431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bc657-881f-46aa-911e-77548a40bc6e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:53.470708 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bc657-881f-46aa-911e-77548a40bc6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:53.471442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bc657-881f-46aa-911e-77548a40bc6e None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=454fb811f6e34e1d881d4948ede058b6) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:53.471753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bc657-881f-46aa-911e-77548a40bc6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:53.473250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bc657-881f-46aa-911e-77548a40bc6e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:53.480243 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 307/650] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:53 2022] GET /identity/v3/role_assignments?user.id=454fb811f6e34e1d881d4948ede058b6&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:53.481126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bc657-881f-46aa-911e-77548a40bc6e None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:53.519712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 344/651] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:53.521149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67144e-680c-4c97-9411-9c4b2edc4862 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:53.596862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:53.630430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bc657-881f-46aa-911e-77548a40bc6e None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:53.657847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67144e-680c-4c97-9411-9c4b2edc4862 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:53.675345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bd9ac-28c0-44fb-a9d4-f44c3203d1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:53.676403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bd9ac-28c0-44fb-a9d4-f44c3203d1bf None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:53.676872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bd9ac-28c0-44fb-a9d4-f44c3203d1bf None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:53.677422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bd9ac-28c0-44fb-a9d4-f44c3203d1bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:53.690835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bd9ac-28c0-44fb-a9d4-f44c3203d1bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-312bd9ac-28c0-44fb-a9d4-f44c3203d1bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:53.698256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:53.699278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:53.700123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:53.701581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:53.701741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70473808-6a81-4789-a581-8c17d2443b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:53.707409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 308/652] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:53.707598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473808-6a81-4789-a581-8c17d2443b77 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:53.707598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473808-6a81-4789-a581-8c17d2443b77 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:53.707598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473808-6a81-4789-a581-8c17d2443b77 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:53.707789 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bd9ac-28c0-44fb-a9d4-f44c3203d1bf None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:53.711365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70473808-6a81-4789-a581-8c17d2443b77 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:53.717860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70473808-6a81-4789-a581-8c17d2443b77 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-70473808-6a81-4789-a581-8c17d2443b77 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:53.726766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:53.727843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 345/653] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 11:40:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:53.729419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70473808-6a81-4789-a581-8c17d2443b77 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:53.864305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bd9ac-28c0-44fb-a9d4-f44c3203d1bf None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:53.884888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70473808-6a81-4789-a581-8c17d2443b77 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:53.897122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:53.922098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ebe79-ba5e-4355-9fee-30fc64cf801c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:53.923277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ebe79-ba5e-4355-9fee-30fc64cf801c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:53.923777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ebe79-ba5e-4355-9fee-30fc64cf801c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:53.924292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ebe79-ba5e-4355-9fee-30fc64cf801c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:53.930130 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906598aa-d099-497a-ba16-5e2a844b248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:53.930745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906598aa-d099-497a-ba16-5e2a844b248f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:53.931023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906598aa-d099-497a-ba16-5e2a844b248f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:53.931345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906598aa-d099-497a-ba16-5e2a844b248f None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:53.932595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906598aa-d099-497a-ba16-5e2a844b248f None admin] RBAC: Adding query filter params (name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:53.932898 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906598aa-d099-497a-ba16-5e2a844b248f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:53.934341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906598aa-d099-497a-ba16-5e2a844b248f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:53.940092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ebe79-ba5e-4355-9fee-30fc64cf801c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:53.941970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ebe79-ba5e-4355-9fee-30fc64cf801c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:53.951362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 346/654] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:53.951362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906598aa-d099-497a-ba16-5e2a844b248f None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-933ebe79-ba5e-4355-9fee-30fc64cf801c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:53.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:53.965428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:53.966380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:53.966380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:53.966380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:53.972939 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 309/655] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:53.974277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ebe79-ba5e-4355-9fee-30fc64cf801c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:54.035263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:54.093902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906598aa-d099-497a-ba16-5e2a844b248f None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:54.107951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ebe79-ba5e-4355-9fee-30fc64cf801c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:54.140990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84022080-80a8-4e92-ab58-c4972e5b60a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:54.141776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84022080-80a8-4e92-ab58-c4972e5b60a7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:54.142168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84022080-80a8-4e92-ab58-c4972e5b60a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:54.142446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84022080-80a8-4e92-ab58-c4972e5b60a7 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:54.143344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84022080-80a8-4e92-ab58-c4972e5b60a7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:54.143647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84022080-80a8-4e92-ab58-c4972e5b60a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:54.145491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84022080-80a8-4e92-ab58-c4972e5b60a7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:54.145730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bb1df-d643-4ff3-b742-1ddaf2054ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:54.149988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bb1df-d643-4ff3-b742-1ddaf2054ff5 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:54.149988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bb1df-d643-4ff3-b742-1ddaf2054ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:54.149988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bb1df-d643-4ff3-b742-1ddaf2054ff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:54.160053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bb1df-d643-4ff3-b742-1ddaf2054ff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:54.162198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bb1df-d643-4ff3-b742-1ddaf2054ff5 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:54.168278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 347/656] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 11:40:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:54.168278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84022080-80a8-4e92-ab58-c4972e5b60a7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-093bb1df-d643-4ff3-b742-1ddaf2054ff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:54.180733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:54.182519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:54.189018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:54.189018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:54.189018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:54.189018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 310/657] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:54.189018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8482ed9-67ee-4b3f-bbd8-aa91cf81dd6f None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:54.189018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8482ed9-67ee-4b3f-bbd8-aa91cf81dd6f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:54.189018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8482ed9-67ee-4b3f-bbd8-aa91cf81dd6f None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:54.190000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 311/658] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:54.191209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8482ed9-67ee-4b3f-bbd8-aa91cf81dd6f None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:54.202993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:54.286980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84022080-80a8-4e92-ab58-c4972e5b60a7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:54.315488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:54.323582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8482ed9-67ee-4b3f-bbd8-aa91cf81dd6f None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:54.328604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a02b6-34cf-4e3c-8d2a-e327b2ff3d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:54.328919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a02b6-34cf-4e3c-8d2a-e327b2ff3d1b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:54.329162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a02b6-34cf-4e3c-8d2a-e327b2ff3d1b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:54.329441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a02b6-34cf-4e3c-8d2a-e327b2ff3d1b None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:54.330085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a02b6-34cf-4e3c-8d2a-e327b2ff3d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:54.330388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a02b6-34cf-4e3c-8d2a-e327b2ff3d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:54.331772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a02b6-34cf-4e3c-8d2a-e327b2ff3d1b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:54.346515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 348/659] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:54.347214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a02b6-34cf-4e3c-8d2a-e327b2ff3d1b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:54.369403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a126fe-aea9-4c63-8848-edf91f3efb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:54.370018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a126fe-aea9-4c63-8848-edf91f3efb9d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:54.370299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a126fe-aea9-4c63-8848-edf91f3efb9d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:54.370575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a126fe-aea9-4c63-8848-edf91f3efb9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:54.447806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a126fe-aea9-4c63-8848-edf91f3efb9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:54.449217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a126fe-aea9-4c63-8848-edf91f3efb9d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:54.451737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:54.466908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a02b6-34cf-4e3c-8d2a-e327b2ff3d1b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:54.509356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cf76b-4688-4f54-9b19-d2130847fe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:54.510927 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cf76b-4688-4f54-9b19-d2130847fe47 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:54.511205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cf76b-4688-4f54-9b19-d2130847fe47 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:54.511477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cf76b-4688-4f54-9b19-d2130847fe47 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:54.512536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cf76b-4688-4f54-9b19-d2130847fe47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:54.512901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cf76b-4688-4f54-9b19-d2130847fe47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:54.515046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cf76b-4688-4f54-9b19-d2130847fe47 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:54.527662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 349/660] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:54.533646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068efc61-e3c9-4e8b-9ef5-ed2272ba8943 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:54.533932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068efc61-e3c9-4e8b-9ef5-ed2272ba8943 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:54.534211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068efc61-e3c9-4e8b-9ef5-ed2272ba8943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:54.554571 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-068efc61-e3c9-4e8b-9ef5-ed2272ba8943 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:54.556029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-068efc61-e3c9-4e8b-9ef5-ed2272ba8943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:54.562829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:54.570991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 312/661] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:54.573342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a126fe-aea9-4c63-8848-edf91f3efb9d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:54.651611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 350/662] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:54.656216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068efc61-e3c9-4e8b-9ef5-ed2272ba8943 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:54.685087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a126fe-aea9-4c63-8848-edf91f3efb9d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:54.729927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4acce-391f-4d7b-992f-5a1e656f5b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:54.730923 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4acce-391f-4d7b-992f-5a1e656f5b1b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:54.731349 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4acce-391f-4d7b-992f-5a1e656f5b1b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:54.731786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4acce-391f-4d7b-992f-5a1e656f5b1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:54.734876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:54.741062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4acce-391f-4d7b-992f-5a1e656f5b1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:54.743033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4acce-391f-4d7b-992f-5a1e656f5b1b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:54.758937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068efc61-e3c9-4e8b-9ef5-ed2272ba8943 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-34b4acce-391f-4d7b-992f-5a1e656f5b1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:54.760246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:54.761507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:54.762619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:54.762619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:54.762619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:54.762619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:54.762619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:54.762619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:54.762619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:54.762619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:54.765798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 313/663] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:54.766922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4acce-391f-4d7b-992f-5a1e656f5b1b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:54.818977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88abbe4-de6e-487d-a11a-9b218c0623c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:54.819775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88abbe4-de6e-487d-a11a-9b218c0623c8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:54.820192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88abbe4-de6e-487d-a11a-9b218c0623c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:54.820650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88abbe4-de6e-487d-a11a-9b218c0623c8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:54.830739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88abbe4-de6e-487d-a11a-9b218c0623c8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:54.833487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88abbe4-de6e-487d-a11a-9b218c0623c8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-f88abbe4-de6e-487d-a11a-9b218c0623c8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:54.849397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:54.850610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:54.852019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:54.852019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:54.852019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:54.852019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:54.852019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:54.852019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:54.852019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:54.852019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:54.852019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 11:40:54.852019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:54.852678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 351/664] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:54.853658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88abbe4-de6e-487d-a11a-9b218c0623c8 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:54.861431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:54.922641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4acce-391f-4d7b-992f-5a1e656f5b1b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:54.964956 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1bf9c7-f319-4a69-aea6-a1e2b9fac55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:54.965605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf9c7-f319-4a69-aea6-a1e2b9fac55a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:54.966280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf9c7-f319-4a69-aea6-a1e2b9fac55a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:54.966572 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf9c7-f319-4a69-aea6-a1e2b9fac55a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:54.982714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bf9c7-f319-4a69-aea6-a1e2b9fac55a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:54.984662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bf9c7-f319-4a69-aea6-a1e2b9fac55a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:54.990587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88abbe4-de6e-487d-a11a-9b218c0623c8 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-de1bf9c7-f319-4a69-aea6-a1e2b9fac55a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:55.002490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:55.003554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:55.004900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:55.004900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:55.004900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:55.004900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 314/665] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:55.004900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1bf9c7-f319-4a69-aea6-a1e2b9fac55a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:55.031246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334cd64-c784-4a39-96ce-ce0bb49f37fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:55.031862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cd64-c784-4a39-96ce-ce0bb49f37fc None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:55.032143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cd64-c784-4a39-96ce-ce0bb49f37fc None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:55.032448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cd64-c784-4a39-96ce-ce0bb49f37fc None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:55.033155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334cd64-c784-4a39-96ce-ce0bb49f37fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:55.033461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334cd64-c784-4a39-96ce-ce0bb49f37fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:55.034904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334cd64-c784-4a39-96ce-ce0bb49f37fc None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:55.045682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:55.051693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 352/666] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:55.052465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334cd64-c784-4a39-96ce-ce0bb49f37fc None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:55.129257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1bf9c7-f319-4a69-aea6-a1e2b9fac55a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:55.139129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:55.165950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf141124-ad88-4f83-8755-9420b58b2dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:55.166528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334cd64-c784-4a39-96ce-ce0bb49f37fc None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:55.167441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf141124-ad88-4f83-8755-9420b58b2dc2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:55.167929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf141124-ad88-4f83-8755-9420b58b2dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:55.168467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf141124-ad88-4f83-8755-9420b58b2dc2 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:55.169533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf141124-ad88-4f83-8755-9420b58b2dc2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:55.169835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf141124-ad88-4f83-8755-9420b58b2dc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:55.171313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf141124-ad88-4f83-8755-9420b58b2dc2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:55.191734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 315/667] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Oct 4 11:40:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:55.192653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf141124-ad88-4f83-8755-9420b58b2dc2 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:55.221002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa93ee-5a70-4e46-959e-b0ddb2014712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:55.221651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa93ee-5a70-4e46-959e-b0ddb2014712 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:55.221958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa93ee-5a70-4e46-959e-b0ddb2014712 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:55.222232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa93ee-5a70-4e46-959e-b0ddb2014712 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:55.235223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa93ee-5a70-4e46-959e-b0ddb2014712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-a0aa93ee-5a70-4e46-959e-b0ddb2014712 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:40:55.241015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:55.242296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 353/668] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:40:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:55.244103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa93ee-5a70-4e46-959e-b0ddb2014712 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:55.317699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf141124-ad88-4f83-8755-9420b58b2dc2 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:55.317985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:55.357567 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0ad4-1340-414a-82ac-8da6937674d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:55.358188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0ad4-1340-414a-82ac-8da6937674d4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:55.358463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0ad4-1340-414a-82ac-8da6937674d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:55.358734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0ad4-1340-414a-82ac-8da6937674d4 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:55.359379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c0ad4-1340-414a-82ac-8da6937674d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:55.359679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c0ad4-1340-414a-82ac-8da6937674d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:55.361112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c0ad4-1340-414a-82ac-8da6937674d4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:55.371004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa93ee-5a70-4e46-959e-b0ddb2014712 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:55.375315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 316/669] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:55.376435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0ad4-1340-414a-82ac-8da6937674d4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:55.409915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aa68c-ff7b-46a4-b22c-5e304e51c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:55.410662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa68c-ff7b-46a4-b22c-5e304e51c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:55.410960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa68c-ff7b-46a4-b22c-5e304e51c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:55.411310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa68c-ff7b-46a4-b22c-5e304e51c8f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:55.412105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24aa68c-ff7b-46a4-b22c-5e304e51c8f6 None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=496a2e8bf591437888ea82161ca4d5ee) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:55.412446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24aa68c-ff7b-46a4-b22c-5e304e51c8f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:55.414005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24aa68c-ff7b-46a4-b22c-5e304e51c8f6 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:55.419005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:55.422064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 354/670] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:55 2022] GET /identity/v3/role_assignments?user.id=496a2e8bf591437888ea82161ca4d5ee&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:55.423101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aa68c-ff7b-46a4-b22c-5e304e51c8f6 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:55.535268 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0ad4-1340-414a-82ac-8da6937674d4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:55.577053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bd79e-e689-451d-b68c-9a35587deac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:55.577841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bd79e-e689-451d-b68c-9a35587deac2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:55.578230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bd79e-e689-451d-b68c-9a35587deac2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:55.578626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bd79e-e689-451d-b68c-9a35587deac2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:55.608433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aa68c-ff7b-46a4-b22c-5e304e51c8f6 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:55.655712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:55.670745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bd79e-e689-451d-b68c-9a35587deac2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:55.672698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bd79e-e689-451d-b68c-9a35587deac2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:55.684122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73584997-ef7b-4ed2-be59-f363156a404b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:55.686902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73584997-ef7b-4ed2-be59-f363156a404b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:55.687334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73584997-ef7b-4ed2-be59-f363156a404b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:55.687747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73584997-ef7b-4ed2-be59-f363156a404b None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:55.690761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73584997-ef7b-4ed2-be59-f363156a404b None admin] RBAC: Adding query filter params (name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:55.690761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73584997-ef7b-4ed2-be59-f363156a404b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:55.694347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73584997-ef7b-4ed2-be59-f363156a404b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:55.709758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 355/671] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:40:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:55.711067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73584997-ef7b-4ed2-be59-f363156a404b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:55.715182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:55.862636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 317/672] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:55.864120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bd79e-e689-451d-b68c-9a35587deac2 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:55.899588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73584997-ef7b-4ed2-be59-f363156a404b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:55.940430 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f76ce-eef7-4fe1-b615-df7e3a2b8cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:55.941071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f76ce-eef7-4fe1-b615-df7e3a2b8cd7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:55.941822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f76ce-eef7-4fe1-b615-df7e3a2b8cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:55.941822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f76ce-eef7-4fe1-b615-df7e3a2b8cd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:55.950538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f76ce-eef7-4fe1-b615-df7e3a2b8cd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:55.952023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f76ce-eef7-4fe1-b615-df7e3a2b8cd7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-458f76ce-eef7-4fe1-b615-df7e3a2b8cd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:55.966224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:55.967294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:55.968238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:55.968238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:55.968238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:55.968238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:55.968238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:55.968238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:55.968238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:55.968238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:55.972709 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 356/673] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:55.973658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f76ce-eef7-4fe1-b615-df7e3a2b8cd7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:55.996026 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bd79e-e689-451d-b68c-9a35587deac2 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:56.026539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:56.032917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:56.036409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd679b22-030f-4339-a8b7-3f45481be6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:56.037257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679b22-030f-4339-a8b7-3f45481be6e3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:56.037693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679b22-030f-4339-a8b7-3f45481be6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:56.037976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679b22-030f-4339-a8b7-3f45481be6e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:56.050686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd679b22-030f-4339-a8b7-3f45481be6e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:56.052434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd679b22-030f-4339-a8b7-3f45481be6e3 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-fd679b22-030f-4339-a8b7-3f45481be6e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:56.071148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:56.072656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:56.073973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:56.073973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:56.073973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:56.077680 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 318/674] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:56.078304 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fe5eb-c1cc-42f7-a4c5-fa4bf914efc3 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:56.078585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fe5eb-c1cc-42f7-a4c5-fa4bf914efc3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:56.078957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fe5eb-c1cc-42f7-a4c5-fa4bf914efc3 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:56.080518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 319/675] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:56.081248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fe5eb-c1cc-42f7-a4c5-fa4bf914efc3 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:56.115801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f76ce-eef7-4fe1-b615-df7e3a2b8cd7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:56.161042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddf2ad-aaa9-4f4f-bf1d-b06f4da130f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:56.161688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf2ad-aaa9-4f4f-bf1d-b06f4da130f1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:56.161964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf2ad-aaa9-4f4f-bf1d-b06f4da130f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:56.162247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf2ad-aaa9-4f4f-bf1d-b06f4da130f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:56.174729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddf2ad-aaa9-4f4f-bf1d-b06f4da130f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:56.177605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddf2ad-aaa9-4f4f-bf1d-b06f4da130f1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-edddf2ad-aaa9-4f4f-bf1d-b06f4da130f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:56.207195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:56.208413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:56.209477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:56.209477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:56.209477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:56.217183 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 357/676] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:56.217693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fe5eb-c1cc-42f7-a4c5-fa4bf914efc3 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:56.218807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddf2ad-aaa9-4f4f-bf1d-b06f4da130f1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:56.255409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b0229-dcc1-4d56-8e5d-7c8d905ef4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:56.256020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b0229-dcc1-4d56-8e5d-7c8d905ef4a6 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:56.256329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b0229-dcc1-4d56-8e5d-7c8d905ef4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:56.256596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b0229-dcc1-4d56-8e5d-7c8d905ef4a6 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:56.257285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b0229-dcc1-4d56-8e5d-7c8d905ef4a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:56.257590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b0229-dcc1-4d56-8e5d-7c8d905ef4a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:56.259063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b0229-dcc1-4d56-8e5d-7c8d905ef4a6 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:56.278029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 320/677] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:56.278029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d561ab-c3fd-4ae5-8bae-e5830573cb7f None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:56.278029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d561ab-c3fd-4ae5-8bae-e5830573cb7f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:56.278029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d561ab-c3fd-4ae5-8bae-e5830573cb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:56.297336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-85d561ab-c3fd-4ae5-8bae-e5830573cb7f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:56.298849 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-85d561ab-c3fd-4ae5-8bae-e5830573cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:56.333334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:56.359194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:56.366204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddf2ad-aaa9-4f4f-bf1d-b06f4da130f1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:56.389092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 321/678] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:56.389092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d561ab-c3fd-4ae5-8bae-e5830573cb7f None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:56.409142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d992df0-8bba-423a-9352-9127fff75aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:56.410053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d992df0-8bba-423a-9352-9127fff75aa0 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:56.410497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d992df0-8bba-423a-9352-9127fff75aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:56.410790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d992df0-8bba-423a-9352-9127fff75aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:56.411411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d992df0-8bba-423a-9352-9127fff75aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:56.411713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d992df0-8bba-423a-9352-9127fff75aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:56.413121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d992df0-8bba-423a-9352-9127fff75aa0 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:56.426629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 358/679] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:56.427298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d992df0-8bba-423a-9352-9127fff75aa0 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:56.524415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d561ab-c3fd-4ae5-8bae-e5830573cb7f None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:56.549614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d992df0-8bba-423a-9352-9127fff75aa0 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:56.572989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135f205-fde6-435b-a720-35326ced2d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:56.573696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135f205-fde6-435b-a720-35326ced2d9f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:56.574618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135f205-fde6-435b-a720-35326ced2d9f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:56.574867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135f205-fde6-435b-a720-35326ced2d9f None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:56.575527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135f205-fde6-435b-a720-35326ced2d9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:56.576108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135f205-fde6-435b-a720-35326ced2d9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:56.577903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135f205-fde6-435b-a720-35326ced2d9f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:56.590765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a14777-feeb-495e-9ce0-e10018341faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:56.591501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a14777-feeb-495e-9ce0-e10018341faa None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:56.591871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a14777-feeb-495e-9ce0-e10018341faa None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:56.592370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a14777-feeb-495e-9ce0-e10018341faa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:56.593269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a14777-feeb-495e-9ce0-e10018341faa None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=ed14c1bc105d4c5cade5813edc1f0dac) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:56.593684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a14777-feeb-495e-9ce0-e10018341faa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:56.595423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a14777-feeb-495e-9ce0-e10018341faa None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:56.598100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 322/680] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:56.599049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135f205-fde6-435b-a720-35326ced2d9f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:56.602974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 359/681] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:56 2022] GET /identity/v3/role_assignments?user.id=ed14c1bc105d4c5cade5813edc1f0dac&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:56.604586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a14777-feeb-495e-9ce0-e10018341faa None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:56.632098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:56.643390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:56.733530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a14777-feeb-495e-9ce0-e10018341faa None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:56.777844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa346f07-a9a3-438e-b150-622108c75add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:56.778065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa346f07-a9a3-438e-b150-622108c75add None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:56.778224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa346f07-a9a3-438e-b150-622108c75add None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:56.778501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa346f07-a9a3-438e-b150-622108c75add None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:56.794197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa346f07-a9a3-438e-b150-622108c75add None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:56.797673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa346f07-a9a3-438e-b150-622108c75add None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:56.815650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135f205-fde6-435b-a720-35326ced2d9f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-fa346f07-a9a3-438e-b150-622108c75add None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:56.817828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:56.819009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:56.820160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:56.820160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:56.820160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:56.820160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:56.820160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:56.820160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:56.820160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:56.820160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:56.823888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 360/682] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:56.825081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa346f07-a9a3-438e-b150-622108c75add None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:56.879113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651b3c-38a3-4a29-aa2c-d8ca63887d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:56.880595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651b3c-38a3-4a29-aa2c-d8ca63887d2d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:56.880886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651b3c-38a3-4a29-aa2c-d8ca63887d2d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:56.881241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651b3c-38a3-4a29-aa2c-d8ca63887d2d None admin] PATH_INFO: `/v3/users/swift` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:56.898669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55651b3c-38a3-4a29-aa2c-d8ca63887d2d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:56.901015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55651b3c-38a3-4a29-aa2c-d8ca63887d2d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-55651b3c-38a3-4a29-aa2c-d8ca63887d2d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:40:56.913751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:56.914851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:56.915908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:56.915908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:56.915908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:40:56.915908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:40:56.915908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:40:56.915908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:40:56.915908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:40:56.915908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:40:56.915908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:40:56.915908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:56.917906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 323/683] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:40:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:56.919001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651b3c-38a3-4a29-aa2c-d8ca63887d2d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:56.957437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:56.995497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa346f07-a9a3-438e-b150-622108c75add None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:57.042012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61cb86f-d0f9-4e81-b313-4caddcf8c373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:57.042806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cb86f-d0f9-4e81-b313-4caddcf8c373 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:57.043121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cb86f-d0f9-4e81-b313-4caddcf8c373 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:57.043400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cb86f-d0f9-4e81-b313-4caddcf8c373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:57.043970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:57.056877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651b3c-38a3-4a29-aa2c-d8ca63887d2d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:57.061208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61cb86f-d0f9-4e81-b313-4caddcf8c373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:57.062810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61cb86f-d0f9-4e81-b313-4caddcf8c373 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-b61cb86f-d0f9-4e81-b313-4caddcf8c373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:57.084390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:57.085428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:57.086370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:57.086370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:57.086370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:57.086370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 361/684] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:57.086657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61cb86f-d0f9-4e81-b313-4caddcf8c373 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:57.112035 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef01b3-fbfa-4615-836c-f0595f7c72e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:57.113512 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef01b3-fbfa-4615-836c-f0595f7c72e9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:57.114113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef01b3-fbfa-4615-836c-f0595f7c72e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:57.114570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef01b3-fbfa-4615-836c-f0595f7c72e9 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:57.115751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef01b3-fbfa-4615-836c-f0595f7c72e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:57.116323 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef01b3-fbfa-4615-836c-f0595f7c72e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:57.118178 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef01b3-fbfa-4615-836c-f0595f7c72e9 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:57.138416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 324/685] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:57.140527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef01b3-fbfa-4615-836c-f0595f7c72e9 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:57.241731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61cb86f-d0f9-4e81-b313-4caddcf8c373 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:57.273746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef01b3-fbfa-4615-836c-f0595f7c72e9 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:57.287315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:57.295243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99ece1-8242-465b-ad19-8cbf8591ddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:57.304582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99ece1-8242-465b-ad19-8cbf8591ddac None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:57.304582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99ece1-8242-465b-ad19-8cbf8591ddac None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:57.304582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99ece1-8242-465b-ad19-8cbf8591ddac None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:57.304582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99ece1-8242-465b-ad19-8cbf8591ddac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:57.304582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99ece1-8242-465b-ad19-8cbf8591ddac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:57.304582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99ece1-8242-465b-ad19-8cbf8591ddac None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:57.312012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d182e-ee21-49b8-9036-ec187650f941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:57.312853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d182e-ee21-49b8-9036-ec187650f941 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:57.313206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d182e-ee21-49b8-9036-ec187650f941 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:57.313563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d182e-ee21-49b8-9036-ec187650f941 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:57.314323 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d182e-ee21-49b8-9036-ec187650f941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:57.324197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d182e-ee21-49b8-9036-ec187650f941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:57.324197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d182e-ee21-49b8-9036-ec187650f941 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:57.324764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 362/686] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 11:40:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:57.325629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99ece1-8242-465b-ad19-8cbf8591ddac None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:57.347892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 325/687] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:57.349533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d182e-ee21-49b8-9036-ec187650f941 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:57.380433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:57.470746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99ece1-8242-465b-ad19-8cbf8591ddac None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:57.487393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d182e-ee21-49b8-9036-ec187650f941 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:57.520110 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0a959-53e8-4f9a-918d-c8d0a54614f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:57.520773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0a959-53e8-4f9a-918d-c8d0a54614f7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:57.521051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0a959-53e8-4f9a-918d-c8d0a54614f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:57.521324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0a959-53e8-4f9a-918d-c8d0a54614f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:57.522353 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0a959-53e8-4f9a-918d-c8d0a54614f7 None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=073e43c049224af29e9f6d426ea47e84) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:57.522669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0a959-53e8-4f9a-918d-c8d0a54614f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:57.525031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0a959-53e8-4f9a-918d-c8d0a54614f7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:57.533857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260f88f-a39c-4a36-b686-0cee93d25e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:57.534483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260f88f-a39c-4a36-b686-0cee93d25e10 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:57.534763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260f88f-a39c-4a36-b686-0cee93d25e10 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:57.535034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260f88f-a39c-4a36-b686-0cee93d25e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:57.544210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 363/688] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:57 2022] GET /identity/v3/role_assignments?user.id=073e43c049224af29e9f6d426ea47e84&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:57.544210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0a959-53e8-4f9a-918d-c8d0a54614f7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:57.671856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:57.686509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260f88f-a39c-4a36-b686-0cee93d25e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:57.689371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260f88f-a39c-4a36-b686-0cee93d25e10 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:57.702118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:57.831455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 326/689] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:40:57.837216 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260f88f-a39c-4a36-b686-0cee93d25e10 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:57.858601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0a959-53e8-4f9a-918d-c8d0a54614f7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:57.904324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d080cc-9580-4718-a5cf-f6fb12ec0219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:57.905351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d080cc-9580-4718-a5cf-f6fb12ec0219 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:57.905837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d080cc-9580-4718-a5cf-f6fb12ec0219 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:57.906326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d080cc-9580-4718-a5cf-f6fb12ec0219 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:57.907466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d080cc-9580-4718-a5cf-f6fb12ec0219 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:57.910276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d080cc-9580-4718-a5cf-f6fb12ec0219 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:57.977857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260f88f-a39c-4a36-b686-0cee93d25e10 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:57.990248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:58.001561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 364/690] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:40:57 2022] POST /identity/v3/users => generated 254 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:40:58.002833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7322357-f7c8-441a-a876-3c31a1c5c07f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:58.003127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7322357-f7c8-441a-a876-3c31a1c5c07f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:58.003409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7322357-f7c8-441a-a876-3c31a1c5c07f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:58.012200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 365/691] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:58.020968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb9537-c0d8-4379-bbbd-e5c38b205fff None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:58.021424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb9537-c0d8-4379-bbbd-e5c38b205fff None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:58.026933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb9537-c0d8-4379-bbbd-e5c38b205fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:58.038798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6eeec-6862-4e8c-b3d9-1d9e090c80a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:58.039601 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6eeec-6862-4e8c-b3d9-1d9e090c80a6 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:58.040069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6eeec-6862-4e8c-b3d9-1d9e090c80a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:58.040585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6eeec-6862-4e8c-b3d9-1d9e090c80a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:58.056082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6eeec-6862-4e8c-b3d9-1d9e090c80a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:58.056981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-beeb9537-c0d8-4379-bbbd-e5c38b205fff None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:58.057868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6eeec-6862-4e8c-b3d9-1d9e090c80a6 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:58.058706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-beeb9537-c0d8-4379-bbbd-e5c38b205fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-56a6eeec-6862-4e8c-b3d9-1d9e090c80a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:40:58.087242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:58.088460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:40:58.089388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:40:58.089388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:40:58.089388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:40:58.090270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 327/692] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:58.097931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6eeec-6862-4e8c-b3d9-1d9e090c80a6 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:58.159359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 366/693] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:58.264523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6eeec-6862-4e8c-b3d9-1d9e090c80a6 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:58.303683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2a20d-bdd2-4a04-82ee-ce942818a0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:58.304323 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2a20d-bdd2-4a04-82ee-ce942818a0e0 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:58.304605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2a20d-bdd2-4a04-82ee-ce942818a0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:58.304879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2a20d-bdd2-4a04-82ee-ce942818a0e0 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:58.305517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2a20d-bdd2-4a04-82ee-ce942818a0e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:58.305821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2a20d-bdd2-4a04-82ee-ce942818a0e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:58.307325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2a20d-bdd2-4a04-82ee-ce942818a0e0 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:58.320608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 328/694] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:58.328420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb9537-c0d8-4379-bbbd-e5c38b205fff None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:58.379566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:58.445337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb9537-c0d8-4379-bbbd-e5c38b205fff None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:58.484184 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc253275-8366-4ed9-b72f-2cded322c178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:58.484876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc253275-8366-4ed9-b72f-2cded322c178 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:58.485165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc253275-8366-4ed9-b72f-2cded322c178 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:58.485443 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc253275-8366-4ed9-b72f-2cded322c178 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:58.493133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc253275-8366-4ed9-b72f-2cded322c178 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:58.495132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc253275-8366-4ed9-b72f-2cded322c178 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-bc253275-8366-4ed9-b72f-2cded322c178 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:40:58.507548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:58.508556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:58.509545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:40:58.509545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:40:58.509545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:40:58.509545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:40:58.509545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:40:58.509545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:40:58.509545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:40:58.509545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:58.509545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 367/695] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:40:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:40:58.518509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2a20d-bdd2-4a04-82ee-ce942818a0e0 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:58.544632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:58.655277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2a20d-bdd2-4a04-82ee-ce942818a0e0 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:58.725334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134823b-3331-44f8-a1b5-ff60c566dfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:58.726331 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134823b-3331-44f8-a1b5-ff60c566dfe9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:58.726826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134823b-3331-44f8-a1b5-ff60c566dfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:58.727324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134823b-3331-44f8-a1b5-ff60c566dfe9 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:58.729131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134823b-3331-44f8-a1b5-ff60c566dfe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:58.729672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134823b-3331-44f8-a1b5-ff60c566dfe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:58.732571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134823b-3331-44f8-a1b5-ff60c566dfe9 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:58.752626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc253275-8366-4ed9-b72f-2cded322c178 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:58.754053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 329/696] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:40:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:58.761904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134823b-3331-44f8-a1b5-ff60c566dfe9 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:58.981002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:59.012647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:59.020208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc253275-8366-4ed9-b72f-2cded322c178 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:59.029479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134823b-3331-44f8-a1b5-ff60c566dfe9 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:59.142058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971881f2-0387-4e97-9222-0ccc99c1b12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:59.142692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971881f2-0387-4e97-9222-0ccc99c1b12b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:59.143017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971881f2-0387-4e97-9222-0ccc99c1b12b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:59.144385 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971881f2-0387-4e97-9222-0ccc99c1b12b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:59.144385 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971881f2-0387-4e97-9222-0ccc99c1b12b None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=734d8f03ff2949959cbfce98212a3ac6) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:59.147887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971881f2-0387-4e97-9222-0ccc99c1b12b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:59.147887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971881f2-0387-4e97-9222-0ccc99c1b12b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:59.154699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 330/697] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:40:58 2022] GET /identity/v3/role_assignments?user.id=734d8f03ff2949959cbfce98212a3ac6&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 608 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:40:59.169152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f9d42-f3cc-4eda-9e12-cd52df34bde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:59.169877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f9d42-f3cc-4eda-9e12-cd52df34bde0 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:59.170167 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f9d42-f3cc-4eda-9e12-cd52df34bde0 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:59.170449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f9d42-f3cc-4eda-9e12-cd52df34bde0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:59.171109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f9d42-f3cc-4eda-9e12-cd52df34bde0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:59.172603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f9d42-f3cc-4eda-9e12-cd52df34bde0 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-1a9f9d42-f3cc-4eda-9e12-cd52df34bde0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:40:59.202834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:40:59.203926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:40:59.204828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 368/698] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Oct 4 11:40:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:40:59.211649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971881f2-0387-4e97-9222-0ccc99c1b12b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:59.390481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971881f2-0387-4e97-9222-0ccc99c1b12b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:59.442838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39235ff2-bbc4-46e5-b9e2-36a82f865ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:59.443463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39235ff2-bbc4-46e5-b9e2-36a82f865ed4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:59.443745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39235ff2-bbc4-46e5-b9e2-36a82f865ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:59.444025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39235ff2-bbc4-46e5-b9e2-36a82f865ed4 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:59.444756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39235ff2-bbc4-46e5-b9e2-36a82f865ed4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:59.445067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39235ff2-bbc4-46e5-b9e2-36a82f865ed4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:59.446488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39235ff2-bbc4-46e5-b9e2-36a82f865ed4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:59.456908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 331/699] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 11:40:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:59.462295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f9d42-f3cc-4eda-9e12-cd52df34bde0 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:40:59.496686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf900-b09a-4780-9ad6-5575190313b2 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:59.497029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf900-b09a-4780-9ad6-5575190313b2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:59.497259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf900-b09a-4780-9ad6-5575190313b2 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:59.499005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 332/700] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:40:59.505138 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b6f40-8df9-4ada-9605-17221cb1d597 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:59.505464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b6f40-8df9-4ada-9605-17221cb1d597 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:59.505704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b6f40-8df9-4ada-9605-17221cb1d597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:59.531625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-298b6f40-8df9-4ada-9605-17221cb1d597 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:40:59.533048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-298b6f40-8df9-4ada-9605-17221cb1d597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:40:59.535495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:40:59.613686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f9d42-f3cc-4eda-9e12-cd52df34bde0 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:40:59.633774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 333/701] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:40:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:40:59.666899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429a117-8440-4636-b62f-4bf4154a9d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:40:59.668019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429a117-8440-4636-b62f-4bf4154a9d84 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:40:59.668503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429a117-8440-4636-b62f-4bf4154a9d84 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:40:59.668940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429a117-8440-4636-b62f-4bf4154a9d84 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:40:59.669728 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429a117-8440-4636-b62f-4bf4154a9d84 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:40:59.670190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429a117-8440-4636-b62f-4bf4154a9d84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:40:59.671711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429a117-8440-4636-b62f-4bf4154a9d84 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:40:59.678834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 369/702] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 11:40:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:40:59.952047 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b6f40-8df9-4ada-9605-17221cb1d597 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:00.085089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b6f40-8df9-4ada-9605-17221cb1d597 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:00.141012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1839c-f49f-416c-97a8-7e7be5ae30e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:00.141709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1839c-f49f-416c-97a8-7e7be5ae30e3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:00.141998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1839c-f49f-416c-97a8-7e7be5ae30e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:00.142273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1839c-f49f-416c-97a8-7e7be5ae30e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:00.150536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:00.154856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1839c-f49f-416c-97a8-7e7be5ae30e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-70f1839c-f49f-416c-97a8-7e7be5ae30e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:00.159984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:00.161061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:00.162025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 334/703] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:40:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:00.169562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429a117-8440-4636-b62f-4bf4154a9d84 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:00.221481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:00.338919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429a117-8440-4636-b62f-4bf4154a9d84 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:00.390359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9371375-d4e3-4196-9655-e289495de457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:00.390979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9371375-d4e3-4196-9655-e289495de457 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:00.391259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9371375-d4e3-4196-9655-e289495de457 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:00.391532 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9371375-d4e3-4196-9655-e289495de457 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:00.392227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9371375-d4e3-4196-9655-e289495de457 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:00.392529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9371375-d4e3-4196-9655-e289495de457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:00.393905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9371375-d4e3-4196-9655-e289495de457 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:00.405569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 370/704] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:41:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:00.411237 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1839c-f49f-416c-97a8-7e7be5ae30e3 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:00.575303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1839c-f49f-416c-97a8-7e7be5ae30e3 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:00.633991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c992607-5d6d-4d2e-8074-3a806d249b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:00.634700 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992607-5d6d-4d2e-8074-3a806d249b10 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:00.634985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992607-5d6d-4d2e-8074-3a806d249b10 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:00.635259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992607-5d6d-4d2e-8074-3a806d249b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:00.650469 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c992607-5d6d-4d2e-8074-3a806d249b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:00.652096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c992607-5d6d-4d2e-8074-3a806d249b10 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-6c992607-5d6d-4d2e-8074-3a806d249b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:00.668461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:00.669539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:00.670446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:00.670446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:00.670446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:00.670446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 335/705] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:00.677337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9371375-d4e3-4196-9655-e289495de457 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:00.875543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:00.892668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9371375-d4e3-4196-9655-e289495de457 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:00.973188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eeac78-a593-4895-8363-bf94a03df639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:00.974028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeac78-a593-4895-8363-bf94a03df639 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:00.974318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeac78-a593-4895-8363-bf94a03df639 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:00.974592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeac78-a593-4895-8363-bf94a03df639 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:00.975609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eeac78-a593-4895-8363-bf94a03df639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:00.975997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eeac78-a593-4895-8363-bf94a03df639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:00.977574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eeac78-a593-4895-8363-bf94a03df639 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:01.000918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 371/706] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:01.008398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c992607-5d6d-4d2e-8074-3a806d249b10 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:01.041264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42598e6-5349-4054-9046-8bfde6436ab9 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:01.041744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42598e6-5349-4054-9046-8bfde6436ab9 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:01.042221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42598e6-5349-4054-9046-8bfde6436ab9 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:01.044153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 372/707] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:01.051001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcaa4b-a8c1-4db4-a1ed-71af4417bce1 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:01.052016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcaa4b-a8c1-4db4-a1ed-71af4417bce1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:01.052577 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcaa4b-a8c1-4db4-a1ed-71af4417bce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:01.081073 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-addcaa4b-a8c1-4db4-a1ed-71af4417bce1 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:01.082310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-addcaa4b-a8c1-4db4-a1ed-71af4417bce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:01.100854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:01.178372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c992607-5d6d-4d2e-8074-3a806d249b10 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:01.211693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 373/708] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 11:41:01.215349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326946f-3404-4149-8c4f-8e98e02cb55d None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:01.215873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326946f-3404-4149-8c4f-8e98e02cb55d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:01.216424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326946f-3404-4149-8c4f-8e98e02cb55d None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:01.220431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 374/709] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:01.238826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc88a7-cefa-4944-8864-36720a148f3b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:01.239439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc88a7-cefa-4944-8864-36720a148f3b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:01.240209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc88a7-cefa-4944-8864-36720a148f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:01.264038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04819ce8-e0a7-4e1a-8f59-f9713dbae036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:01.266599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04819ce8-e0a7-4e1a-8f59-f9713dbae036 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:01.266948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04819ce8-e0a7-4e1a-8f59-f9713dbae036 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:01.267230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04819ce8-e0a7-4e1a-8f59-f9713dbae036 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:01.274751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4fdc88a7-cefa-4944-8864-36720a148f3b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:01.280345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4fdc88a7-cefa-4944-8864-36720a148f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:01.284791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04819ce8-e0a7-4e1a-8f59-f9713dbae036 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:01.286396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04819ce8-e0a7-4e1a-8f59-f9713dbae036 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-04819ce8-e0a7-4e1a-8f59-f9713dbae036 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:01.309609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:01.310684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:01.311692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:01.311692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:01.311692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:01.311692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:01.311692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:01.311692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:01.311692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:01.311692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:01.311692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:41:01.311692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:01.314022 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 336/710] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 11:41:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:01.318358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04819ce8-e0a7-4e1a-8f59-f9713dbae036 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:01.433730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 375/711] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:41:01.487419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04819ce8-e0a7-4e1a-8f59-f9713dbae036 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:01.499676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:01.533704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781991f9-86d7-48c0-b6cf-266492929eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:01.534330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781991f9-86d7-48c0-b6cf-266492929eb3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:01.534612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781991f9-86d7-48c0-b6cf-266492929eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:01.534951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781991f9-86d7-48c0-b6cf-266492929eb3 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:01.535692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781991f9-86d7-48c0-b6cf-266492929eb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:01.536002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781991f9-86d7-48c0-b6cf-266492929eb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:01.537761 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781991f9-86d7-48c0-b6cf-266492929eb3 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:01.538349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc88a7-cefa-4944-8864-36720a148f3b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:01.551715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:01.568186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 337/712] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 11:41:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:01.574553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781991f9-86d7-48c0-b6cf-266492929eb3 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:01.657035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc88a7-cefa-4944-8864-36720a148f3b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:01.698991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d379961-6a2c-4fd9-9729-c5643fa9c9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:01.700439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d379961-6a2c-4fd9-9729-c5643fa9c9e8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:01.700784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781991f9-86d7-48c0-b6cf-266492929eb3 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:01.701056 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d379961-6a2c-4fd9-9729-c5643fa9c9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:01.701477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d379961-6a2c-4fd9-9729-c5643fa9c9e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:01.702352 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d379961-6a2c-4fd9-9729-c5643fa9c9e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:01.704743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d379961-6a2c-4fd9-9729-c5643fa9c9e8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-0d379961-6a2c-4fd9-9729-c5643fa9c9e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:01.715609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:01.716974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 11:41:01.718153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:01.725951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 376/713] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:01 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:01.727664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d379961-6a2c-4fd9-9729-c5643fa9c9e8 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:01.742106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41abe3-6f33-4559-82d7-d5f2d332380b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:01.742800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41abe3-6f33-4559-82d7-d5f2d332380b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:01.743133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41abe3-6f33-4559-82d7-d5f2d332380b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:01.743416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41abe3-6f33-4559-82d7-d5f2d332380b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:01.795319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:01.827131 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:01.844043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41abe3-6f33-4559-82d7-d5f2d332380b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:01.846356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41abe3-6f33-4559-82d7-d5f2d332380b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:01.857481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d379961-6a2c-4fd9-9729-c5643fa9c9e8 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:01.912842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127745a-60f4-4abb-88c7-61fbac464243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:01.913460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127745a-60f4-4abb-88c7-61fbac464243 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:01.913738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127745a-60f4-4abb-88c7-61fbac464243 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:01.914054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127745a-60f4-4abb-88c7-61fbac464243 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:01.914657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127745a-60f4-4abb-88c7-61fbac464243 None admin] RBAC: Adding query filter params (name=network) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:01.914946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127745a-60f4-4abb-88c7-61fbac464243 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:01.916328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127745a-60f4-4abb-88c7-61fbac464243 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:01.927204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 377/714] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:01.928065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127745a-60f4-4abb-88c7-61fbac464243 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:02.000042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 338/715] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:41:02.001411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41abe3-6f33-4559-82d7-d5f2d332380b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:02.059872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127745a-60f4-4abb-88c7-61fbac464243 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:02.088957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:02.109767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d914c-69bb-4070-beb2-3290a7bb3e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:02.110527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d914c-69bb-4070-beb2-3290a7bb3e34 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.110983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d914c-69bb-4070-beb2-3290a7bb3e34 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.111389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d914c-69bb-4070-beb2-3290a7bb3e34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.126831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:02.135368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d914c-69bb-4070-beb2-3290a7bb3e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-331d914c-69bb-4070-beb2-3290a7bb3e34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:02.144569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:02.146471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:41:02.149134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:02.151023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 378/716] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:41:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:02.151812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d914c-69bb-4070-beb2-3290a7bb3e34 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:02.166441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41abe3-6f33-4559-82d7-d5f2d332380b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:02.283590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd42b6a-7fbb-4410-992b-c192f3de4530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:02.285330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd42b6a-7fbb-4410-992b-c192f3de4530 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.285865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd42b6a-7fbb-4410-992b-c192f3de4530 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.286286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd42b6a-7fbb-4410-992b-c192f3de4530 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.287092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd42b6a-7fbb-4410-992b-c192f3de4530 None admin] RBAC: Adding query filter params (type=network) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:02.287547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd42b6a-7fbb-4410-992b-c192f3de4530 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:02.289755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd42b6a-7fbb-4410-992b-c192f3de4530 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:02.298011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 339/717] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:02.299183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd42b6a-7fbb-4410-992b-c192f3de4530 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:02.358294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d914c-69bb-4070-beb2-3290a7bb3e34 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:02.445615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47b365-7a2a-459c-82a5-159c5a8ac420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:02.446310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b365-7a2a-459c-82a5-159c5a8ac420 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.446590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b365-7a2a-459c-82a5-159c5a8ac420 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.446936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b365-7a2a-459c-82a5-159c5a8ac420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.463497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47b365-7a2a-459c-82a5-159c5a8ac420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:02.465057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47b365-7a2a-459c-82a5-159c5a8ac420 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-9e47b365-7a2a-459c-82a5-159c5a8ac420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:02.485374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:02.486550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:02.487747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:02.487747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:02.487747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:02.491396 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 379/718] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:02.492745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bf17e-14d0-487a-9f69-4c818317a344 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.493041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bf17e-14d0-487a-9f69-4c818317a344 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.493319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bf17e-14d0-487a-9f69-4c818317a344 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.495164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 380/719] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:02.496021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bf17e-14d0-487a-9f69-4c818317a344 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:02.496513 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd42b6a-7fbb-4410-992b-c192f3de4530 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:02.548077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:02.556046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f55f3-1a31-4cce-8b39-2c2235d776aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:02.562311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f55f3-1a31-4cce-8b39-2c2235d776aa None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.562311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f55f3-1a31-4cce-8b39-2c2235d776aa None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.562311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f55f3-1a31-4cce-8b39-2c2235d776aa None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.562311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f55f3-1a31-4cce-8b39-2c2235d776aa None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:02.562311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f55f3-1a31-4cce-8b39-2c2235d776aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:02.562311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f55f3-1a31-4cce-8b39-2c2235d776aa None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:02.578966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 340/720] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:41:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:02.585541 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:02.586494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea057b-e097-4b20-9b7b-401cac5f023c None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.586967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea057b-e097-4b20-9b7b-401cac5f023c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.587389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea057b-e097-4b20-9b7b-401cac5f023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.607049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-fcea057b-e097-4b20-9b7b-401cac5f023c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:02.608943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-fcea057b-e097-4b20-9b7b-401cac5f023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:02.648824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bf17e-14d0-487a-9f69-4c818317a344 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:02.690474 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de15df9-8b86-4003-bcd6-5f224981de8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:02.691083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de15df9-8b86-4003-bcd6-5f224981de8c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.691358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de15df9-8b86-4003-bcd6-5f224981de8c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.691682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de15df9-8b86-4003-bcd6-5f224981de8c None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.692374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de15df9-8b86-4003-bcd6-5f224981de8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:02.692679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de15df9-8b86-4003-bcd6-5f224981de8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:02.694069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de15df9-8b86-4003-bcd6-5f224981de8c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:02.701156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 341/721] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:41:02.703027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea057b-e097-4b20-9b7b-401cac5f023c None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:02.723615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 381/722] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:02.725793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fcd10-02ee-4aaf-ae78-62639211fd1b None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.726337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fcd10-02ee-4aaf-ae78-62639211fd1b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.726924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fcd10-02ee-4aaf-ae78-62639211fd1b None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.729478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 382/723] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:02.730740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fcd10-02ee-4aaf-ae78-62639211fd1b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:02.861273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea057b-e097-4b20-9b7b-401cac5f023c None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:02.864983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fcd10-02ee-4aaf-ae78-62639211fd1b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:02.878417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:02.890876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:02.902925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91462c9e-1559-41f2-814c-8580eb79904b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:02.903944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91462c9e-1559-41f2-814c-8580eb79904b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.904464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91462c9e-1559-41f2-814c-8580eb79904b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.904934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91462c9e-1559-41f2-814c-8580eb79904b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.906968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dde623-6b4d-458a-a638-656ebc2d6ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:02.907899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dde623-6b4d-458a-a638-656ebc2d6ad5 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.908284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dde623-6b4d-458a-a638-656ebc2d6ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.908705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dde623-6b4d-458a-a638-656ebc2d6ad5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.917384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91462c9e-1559-41f2-814c-8580eb79904b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:02.919150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91462c9e-1559-41f2-814c-8580eb79904b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:02.919421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dde623-6b4d-458a-a638-656ebc2d6ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:02.922547 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dde623-6b4d-458a-a638-656ebc2d6ad5 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-21dde623-6b4d-458a-a638-656ebc2d6ad5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:02.934125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:02.935084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:02.935980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:02.935980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:02.935980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:02.935980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:02.935980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:02.935980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:02.935980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:02.935980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:02.935980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 383/724] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:02.937650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a4378-d999-482c-ba74-2270f54ed7d9 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:02.938002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a4378-d999-482c-ba74-2270f54ed7d9 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:02.938350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a4378-d999-482c-ba74-2270f54ed7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-91462c9e-1559-41f2-814c-8580eb79904b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:02.939290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:02.940230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:02.941224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:02.941224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:02.941224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:02.942834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 342/725] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:02.943985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91462c9e-1559-41f2-814c-8580eb79904b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:02.970808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-dd1a4378-d999-482c-ba74-2270f54ed7d9 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:02.972095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-dd1a4378-d999-482c-ba74-2270f54ed7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:03.071093 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 384/726] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:41:03.072496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a4378-d999-482c-ba74-2270f54ed7d9 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:03.075058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91462c9e-1559-41f2-814c-8580eb79904b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:03.140518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0faa2-071d-4801-b1df-864d31ab0604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:03.141154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0faa2-071d-4801-b1df-864d31ab0604 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:03.141426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0faa2-071d-4801-b1df-864d31ab0604 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:03.141706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0faa2-071d-4801-b1df-864d31ab0604 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:03.142406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0faa2-071d-4801-b1df-864d31ab0604 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:03.142706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0faa2-071d-4801-b1df-864d31ab0604 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:03.144144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0faa2-071d-4801-b1df-864d31ab0604 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:03.164374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 343/727] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:03.166739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0faa2-071d-4801-b1df-864d31ab0604 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:03.198246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:03.201380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a4378-d999-482c-ba74-2270f54ed7d9 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:03.216123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:03.241725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cf85a-5959-49a8-8d24-f702e5867cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:03.242429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cf85a-5959-49a8-8d24-f702e5867cbc None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:03.242771 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cf85a-5959-49a8-8d24-f702e5867cbc None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:03.243596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cf85a-5959-49a8-8d24-f702e5867cbc None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:03.244328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cf85a-5959-49a8-8d24-f702e5867cbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:03.244635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cf85a-5959-49a8-8d24-f702e5867cbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:03.246414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cf85a-5959-49a8-8d24-f702e5867cbc None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:03.263260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 385/728] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:03.264313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cf85a-5959-49a8-8d24-f702e5867cbc None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:03.304878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0faa2-071d-4801-b1df-864d31ab0604 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:03.346944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a513f1-0f8b-4e89-9731-c55350e69a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:03.347687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a513f1-0f8b-4e89-9731-c55350e69a86 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:03.348031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a513f1-0f8b-4e89-9731-c55350e69a86 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:03.348489 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a513f1-0f8b-4e89-9731-c55350e69a86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:03.361441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a513f1-0f8b-4e89-9731-c55350e69a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-73a513f1-0f8b-4e89-9731-c55350e69a86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:03.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:03.368851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:03.369725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:03.373914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 344/729] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:41:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:03.375572 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a513f1-0f8b-4e89-9731-c55350e69a86 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:03.411812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cf85a-5959-49a8-8d24-f702e5867cbc None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:03.450240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5b420-1459-43d8-8009-99426c7abb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:03.451310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5b420-1459-43d8-8009-99426c7abb55 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:03.451591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5b420-1459-43d8-8009-99426c7abb55 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:03.451913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5b420-1459-43d8-8009-99426c7abb55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:03.452643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5b420-1459-43d8-8009-99426c7abb55 None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=ed14c1bc105d4c5cade5813edc1f0dac) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:03.452958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5b420-1459-43d8-8009-99426c7abb55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:03.454390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5b420-1459-43d8-8009-99426c7abb55 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:03.468216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 386/730] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:41:03 2022] GET /identity/v3/role_assignments?user.id=ed14c1bc105d4c5cade5813edc1f0dac&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:03.468216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5b420-1459-43d8-8009-99426c7abb55 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:03.487704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:03.494570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a513f1-0f8b-4e89-9731-c55350e69a86 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:03.508461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:03.535821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74d21d-78f6-4f6d-ba5f-3de70b0097b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:03.536755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d21d-78f6-4f6d-ba5f-3de70b0097b8 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:03.537182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d21d-78f6-4f6d-ba5f-3de70b0097b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:03.537630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d21d-78f6-4f6d-ba5f-3de70b0097b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:03.546654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74d21d-78f6-4f6d-ba5f-3de70b0097b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:03.548468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74d21d-78f6-4f6d-ba5f-3de70b0097b8 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-1a74d21d-78f6-4f6d-ba5f-3de70b0097b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:03.562188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:03.563219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:03.564206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:03.564206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:03.564206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:03.564206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:03.564206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:03.564206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:03.564206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:03.564206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:03.564206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:03.564206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:03.566442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 345/731] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 11:41:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:03.567552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74d21d-78f6-4f6d-ba5f-3de70b0097b8 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:03.617246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5b420-1459-43d8-8009-99426c7abb55 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:03.674448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fc642-e7c4-47fd-975c-104e60487e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:03.675725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fc642-e7c4-47fd-975c-104e60487e37 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:03.676220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fc642-e7c4-47fd-975c-104e60487e37 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:03.676676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fc642-e7c4-47fd-975c-104e60487e37 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:03.677747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fc642-e7c4-47fd-975c-104e60487e37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:03.679364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fc642-e7c4-47fd-975c-104e60487e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:03.681710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fc642-e7c4-47fd-975c-104e60487e37 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:03.692590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 387/732] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:41:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:03.693501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fc642-e7c4-47fd-975c-104e60487e37 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:03.726260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74d21d-78f6-4f6d-ba5f-3de70b0097b8 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:03.773652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bd08c-bf97-4580-ac98-0a95f359c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:03.775342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bd08c-bf97-4580-ac98-0a95f359c1a6 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:03.775839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bd08c-bf97-4580-ac98-0a95f359c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:03.776377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bd08c-bf97-4580-ac98-0a95f359c1a6 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:03.777280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bd08c-bf97-4580-ac98-0a95f359c1a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:03.779058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bd08c-bf97-4580-ac98-0a95f359c1a6 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:03.821493 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:03.841447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fc642-e7c4-47fd-975c-104e60487e37 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:03.885199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 346/733] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:41:03 2022] POST /identity/v3/services => generated 251 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:03.885199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bd08c-bf97-4580-ac98-0a95f359c1a6 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:03.936905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebfd86-3e54-4642-91c0-a615c4ea967d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:03.937716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebfd86-3e54-4642-91c0-a615c4ea967d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:03.938155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebfd86-3e54-4642-91c0-a615c4ea967d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:03.939202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:03.942144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebfd86-3e54-4642-91c0-a615c4ea967d None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:03.944743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebfd86-3e54-4642-91c0-a615c4ea967d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:03.945282 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebfd86-3e54-4642-91c0-a615c4ea967d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:03.950766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebfd86-3e54-4642-91c0-a615c4ea967d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:04.016637 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 388/734] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 11:41:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:04.023026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebfd86-3e54-4642-91c0-a615c4ea967d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:04.128951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bd08c-bf97-4580-ac98-0a95f359c1a6 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:04.204119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebfd86-3e54-4642-91c0-a615c4ea967d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:04.213919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f4783-e327-41ae-aeec-daa978f92f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:04.215033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f4783-e327-41ae-aeec-daa978f92f1a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:04.215550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f4783-e327-41ae-aeec-daa978f92f1a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:04.216061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f4783-e327-41ae-aeec-daa978f92f1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:04.242820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f4783-e327-41ae-aeec-daa978f92f1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:04.246787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f4783-e327-41ae-aeec-daa978f92f1a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:04.251648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f286da1-c316-4f72-ac63-7a82f37adea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:04.252599 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f286da1-c316-4f72-ac63-7a82f37adea0 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:04.253064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f286da1-c316-4f72-ac63-7a82f37adea0 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:04.253546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f286da1-c316-4f72-ac63-7a82f37adea0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:04.270310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-437f4783-e327-41ae-aeec-daa978f92f1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:04.278148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:04.279272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:04.280303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:04.280303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:04.280303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:04.280303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 347/735] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:04.287691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f4783-e327-41ae-aeec-daa978f92f1a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:04.372385 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f286da1-c316-4f72-ac63-7a82f37adea0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:04.379057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f286da1-c316-4f72-ac63-7a82f37adea0 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:04.460637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:04.576815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f4783-e327-41ae-aeec-daa978f92f1a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:04.638453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668335f9-9cb3-44ee-8253-9e816aba595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:04.639065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668335f9-9cb3-44ee-8253-9e816aba595a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:04.639343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668335f9-9cb3-44ee-8253-9e816aba595a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:04.639614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668335f9-9cb3-44ee-8253-9e816aba595a None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:04.640285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668335f9-9cb3-44ee-8253-9e816aba595a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:04.640586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668335f9-9cb3-44ee-8253-9e816aba595a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:04.643275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668335f9-9cb3-44ee-8253-9e816aba595a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:04.660385 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 348/736] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:04.662088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668335f9-9cb3-44ee-8253-9e816aba595a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:04.687432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 389/737] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:04 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 11:41:04.697354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f286da1-c316-4f72-ac63-7a82f37adea0 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:04.777729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:04.846834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668335f9-9cb3-44ee-8253-9e816aba595a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:04.893131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:04.894345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f286da1-c316-4f72-ac63-7a82f37adea0 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:04.905364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26843634-b34b-46b0-92da-e02c73a90193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:04.907983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26843634-b34b-46b0-92da-e02c73a90193 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:04.907983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26843634-b34b-46b0-92da-e02c73a90193 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:04.907983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26843634-b34b-46b0-92da-e02c73a90193 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:04.916851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26843634-b34b-46b0-92da-e02c73a90193 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:04.918439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26843634-b34b-46b0-92da-e02c73a90193 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-26843634-b34b-46b0-92da-e02c73a90193 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:04.935518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:04.936641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:04.937624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 349/738] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:41:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:04.938711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc8d54-a4cc-436e-83f6-39d1e5276bba None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:04.938711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc8d54-a4cc-436e-83f6-39d1e5276bba None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:04.938884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc8d54-a4cc-436e-83f6-39d1e5276bba None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:04.944002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 350/739] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:04.944002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc8d54-a4cc-436e-83f6-39d1e5276bba None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:04.945681 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89362c1-c69c-4681-9ef1-e963b3242203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:04.946574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89362c1-c69c-4681-9ef1-e963b3242203 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:04.947625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89362c1-c69c-4681-9ef1-e963b3242203 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:04.948099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89362c1-c69c-4681-9ef1-e963b3242203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:04.959398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89362c1-c69c-4681-9ef1-e963b3242203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:04.961792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89362c1-c69c-4681-9ef1-e963b3242203 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-b89362c1-c69c-4681-9ef1-e963b3242203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:04.979918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:04.981121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:04.982046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:04.982046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:04.982046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:04.988312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 390/740] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:04.990070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851d9d-4021-4092-84b4-67e73432c0f3 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:04.990576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851d9d-4021-4092-84b4-67e73432c0f3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:04.991055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851d9d-4021-4092-84b4-67e73432c0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:05.013216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-a4851d9d-4021-4092-84b4-67e73432c0f3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:05.015291 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a4851d9d-4021-4092-84b4-67e73432c0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:05.076132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc8d54-a4cc-436e-83f6-39d1e5276bba None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:05.102715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 391/741] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 11:41:05.102715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851d9d-4021-4092-84b4-67e73432c0f3 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:05.152135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:05.154613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8ae45-5045-4d96-b3ab-2ceb3d0f2bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:05.155500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ae45-5045-4d96-b3ab-2ceb3d0f2bf2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:05.155993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ae45-5045-4d96-b3ab-2ceb3d0f2bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:05.156504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ae45-5045-4d96-b3ab-2ceb3d0f2bf2 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:05.157437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8ae45-5045-4d96-b3ab-2ceb3d0f2bf2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:05.157953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8ae45-5045-4d96-b3ab-2ceb3d0f2bf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:05.159666 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8ae45-5045-4d96-b3ab-2ceb3d0f2bf2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:05.198288 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 351/742] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 11:41:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:05.201337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8ae45-5045-4d96-b3ab-2ceb3d0f2bf2 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:05.297676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:05.300004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851d9d-4021-4092-84b4-67e73432c0f3 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:05.350489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df2ac3-8c16-4ec6-9f96-6036aad41b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:05.351520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df2ac3-8c16-4ec6-9f96-6036aad41b48 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:05.352014 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df2ac3-8c16-4ec6-9f96-6036aad41b48 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:05.352555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df2ac3-8c16-4ec6-9f96-6036aad41b48 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:05.353513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df2ac3-8c16-4ec6-9f96-6036aad41b48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:05.354046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df2ac3-8c16-4ec6-9f96-6036aad41b48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:05.356382 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df2ac3-8c16-4ec6-9f96-6036aad41b48 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:05.390694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 392/743] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:05.397646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df2ac3-8c16-4ec6-9f96-6036aad41b48 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:05.436833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8ae45-5045-4d96-b3ab-2ceb3d0f2bf2 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:05.494471 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a0751-08f4-4515-80de-0f9bbd2deb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:05.495333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a0751-08f4-4515-80de-0f9bbd2deb74 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:05.496225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a0751-08f4-4515-80de-0f9bbd2deb74 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:05.496504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a0751-08f4-4515-80de-0f9bbd2deb74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:05.530536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:05.554459 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df2ac3-8c16-4ec6-9f96-6036aad41b48 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:05.589544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a0751-08f4-4515-80de-0f9bbd2deb74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:05.591692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a0751-08f4-4515-80de-0f9bbd2deb74 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:05.603479 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38117a-af0b-4570-9089-38994427eb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:05.604857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38117a-af0b-4570-9089-38994427eb13 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:05.605619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38117a-af0b-4570-9089-38994427eb13 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:05.606231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38117a-af0b-4570-9089-38994427eb13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:05.619900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38117a-af0b-4570-9089-38994427eb13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:05.622863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38117a-af0b-4570-9089-38994427eb13 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-9c38117a-af0b-4570-9089-38994427eb13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:05.641978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:05.642964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:05.643876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:05.643876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:05.643876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:05.643876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:05.643876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:05.643876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:05.643876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:05.643876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:05.643876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 393/744] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:05.644408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38117a-af0b-4570-9089-38994427eb13 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:05.651357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:05.723028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 352/745] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:05 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 11:41:05.724213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a0751-08f4-4515-80de-0f9bbd2deb74 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:05.787074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38117a-af0b-4570-9089-38994427eb13 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:05.852105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1977730-3530-4719-9bae-7f4161224164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:05.853766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1977730-3530-4719-9bae-7f4161224164 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:05.854143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1977730-3530-4719-9bae-7f4161224164 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:05.854427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1977730-3530-4719-9bae-7f4161224164 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:05.855142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1977730-3530-4719-9bae-7f4161224164 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:05.856593 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1977730-3530-4719-9bae-7f4161224164 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:05.860064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a0751-08f4-4515-80de-0f9bbd2deb74 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:05.901054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26489aa1-c6be-4a6a-a775-d976f038a98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:05.901673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26489aa1-c6be-4a6a-a775-d976f038a98b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:05.901951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26489aa1-c6be-4a6a-a775-d976f038a98b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:05.902224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26489aa1-c6be-4a6a-a775-d976f038a98b None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:05.902980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26489aa1-c6be-4a6a-a775-d976f038a98b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:05.903289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26489aa1-c6be-4a6a-a775-d976f038a98b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:05.904805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26489aa1-c6be-4a6a-a775-d976f038a98b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:05.919604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 353/746] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:05.920503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26489aa1-c6be-4a6a-a775-d976f038a98b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:05.936185 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 394/747] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:41:05 2022] POST /identity/v3/services => generated 240 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:05.938342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0bf1f-d356-4482-987e-8eb3fdf1e18f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:05.938869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0bf1f-d356-4482-987e-8eb3fdf1e18f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:05.939147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0bf1f-d356-4482-987e-8eb3fdf1e18f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:05.944959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 395/748] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:05.945592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:05.945851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0bf1f-d356-4482-987e-8eb3fdf1e18f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:05.950755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:06.106147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26489aa1-c6be-4a6a-a775-d976f038a98b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:06.120452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0bf1f-d356-4482-987e-8eb3fdf1e18f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:06.153618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a06126-0246-4cc4-8006-f85a640d6f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:06.154641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a06126-0246-4cc4-8006-f85a640d6f49 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:06.155148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a06126-0246-4cc4-8006-f85a640d6f49 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:06.155690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a06126-0246-4cc4-8006-f85a640d6f49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:06.169262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a06126-0246-4cc4-8006-f85a640d6f49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:06.171529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a06126-0246-4cc4-8006-f85a640d6f49 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:06.173355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f23bde-684f-4e12-b3f8-10256ff1c5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:06.173997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f23bde-684f-4e12-b3f8-10256ff1c5ec None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:06.174334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f23bde-684f-4e12-b3f8-10256ff1c5ec None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:06.174900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f23bde-684f-4e12-b3f8-10256ff1c5ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:06.176127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f23bde-684f-4e12-b3f8-10256ff1c5ec None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=952ec01f4a174b6d94c3afd98f90ba81) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:06.176562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f23bde-684f-4e12-b3f8-10256ff1c5ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:06.178438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f23bde-684f-4e12-b3f8-10256ff1c5ec None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:06.197974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 396/749] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:41:05 2022] GET /identity/v3/role_assignments?user.id=952ec01f4a174b6d94c3afd98f90ba81&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:06.197974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8155c5-84b5-4ae6-ba77-9330ffddbbc8 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:06.197974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8155c5-84b5-4ae6-ba77-9330ffddbbc8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:06.197974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8155c5-84b5-4ae6-ba77-9330ffddbbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-18a06126-0246-4cc4-8006-f85a640d6f49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:06.198406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:06.199290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:06.200740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:06.200740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:06.200740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:06.200740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 354/750] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:06.200740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a06126-0246-4cc4-8006-f85a640d6f49 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:06.230705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9b8155c5-84b5-4ae6-ba77-9330ffddbbc8 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:06.232164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9b8155c5-84b5-4ae6-ba77-9330ffddbbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:06.296725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:06.343280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:06.365429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 397/751] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:06.375575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a06126-0246-4cc4-8006-f85a640d6f49 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:06.453140 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482fdc3-7c1e-47e6-ad9e-0f8332877e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:06.454212 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482fdc3-7c1e-47e6-ad9e-0f8332877e06 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:06.454757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482fdc3-7c1e-47e6-ad9e-0f8332877e06 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:06.455358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482fdc3-7c1e-47e6-ad9e-0f8332877e06 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:06.456409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482fdc3-7c1e-47e6-ad9e-0f8332877e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:06.456957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482fdc3-7c1e-47e6-ad9e-0f8332877e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:06.458764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482fdc3-7c1e-47e6-ad9e-0f8332877e06 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:06.476951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 355/752] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:06.484376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8155c5-84b5-4ae6-ba77-9330ffddbbc8 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:06.573767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c4406-d6be-425d-9a77-94c6f02fe290 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:06.574220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c4406-d6be-425d-9a77-94c6f02fe290 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:06.574618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c4406-d6be-425d-9a77-94c6f02fe290 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:06.576076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 356/753] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:06.583490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda2f2f-7ebb-448d-b163-81bd2cfe0b37 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:06.584658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda2f2f-7ebb-448d-b163-81bd2cfe0b37 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:06.584658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda2f2f-7ebb-448d-b163-81bd2cfe0b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:06.619036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8155c5-84b5-4ae6-ba77-9330ffddbbc8 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:06.620363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-bbda2f2f-7ebb-448d-b163-81bd2cfe0b37 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:06.621829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-bbda2f2f-7ebb-448d-b163-81bd2cfe0b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:06.669690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443643f-3cb6-4ba1-87de-a9e0ff73464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:06.670624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443643f-3cb6-4ba1-87de-a9e0ff73464e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:06.671056 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443643f-3cb6-4ba1-87de-a9e0ff73464e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:06.671477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443643f-3cb6-4ba1-87de-a9e0ff73464e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:06.681611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443643f-3cb6-4ba1-87de-a9e0ff73464e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:06.683454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443643f-3cb6-4ba1-87de-a9e0ff73464e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-b443643f-3cb6-4ba1-87de-a9e0ff73464e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:06.711745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:06.713414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:06.714525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:06.714525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:06.714525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:06.714525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:06.714525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:06.714525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:06.714525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:06.714525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:06.716206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 357/754] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:06.717274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 398/755] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:06.723028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda2f2f-7ebb-448d-b163-81bd2cfe0b37 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:06.787500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:06.823904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443643f-3cb6-4ba1-87de-a9e0ff73464e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:06.849047 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda2f2f-7ebb-448d-b163-81bd2cfe0b37 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:06.888640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07cc48-966d-4aeb-a282-5298940b6646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:06.889620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07cc48-966d-4aeb-a282-5298940b6646 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:06.890088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07cc48-966d-4aeb-a282-5298940b6646 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:06.890557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07cc48-966d-4aeb-a282-5298940b6646 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:06.891474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07cc48-966d-4aeb-a282-5298940b6646 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:06.891984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07cc48-966d-4aeb-a282-5298940b6646 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:06.893624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07cc48-966d-4aeb-a282-5298940b6646 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:06.907205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 358/756] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:06.913326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07cc48-966d-4aeb-a282-5298940b6646 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:06.944554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:07.047899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443643f-3cb6-4ba1-87de-a9e0ff73464e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:07.119941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327b0c-d2a7-4f0e-8692-ac7526234aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:07.121040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327b0c-d2a7-4f0e-8692-ac7526234aed None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:07.121228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07cc48-966d-4aeb-a282-5298940b6646 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:07.121537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327b0c-d2a7-4f0e-8692-ac7526234aed None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:07.122039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327b0c-d2a7-4f0e-8692-ac7526234aed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:07.132538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:07.146653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8327b0c-d2a7-4f0e-8692-ac7526234aed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-b8327b0c-d2a7-4f0e-8692-ac7526234aed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:07.156618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:07.157781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:07.158746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:07.159421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 399/757] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:41:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:07.163928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327b0c-d2a7-4f0e-8692-ac7526234aed None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:07.174725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8cc95-6133-4eda-8c7b-f7c6aa8a026b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:07.175727 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8cc95-6133-4eda-8c7b-f7c6aa8a026b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:07.179517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8cc95-6133-4eda-8c7b-f7c6aa8a026b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:07.179517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8cc95-6133-4eda-8c7b-f7c6aa8a026b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:07.179517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8cc95-6133-4eda-8c7b-f7c6aa8a026b None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=734d8f03ff2949959cbfce98212a3ac6) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:07.179517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8cc95-6133-4eda-8c7b-f7c6aa8a026b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:07.179517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8cc95-6133-4eda-8c7b-f7c6aa8a026b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:07.187039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 359/758] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:41:06 2022] GET /identity/v3/role_assignments?user.id=734d8f03ff2949959cbfce98212a3ac6&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:07.188366 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8cc95-6133-4eda-8c7b-f7c6aa8a026b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:07.374856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327b0c-d2a7-4f0e-8692-ac7526234aed None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:07.385771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8cc95-6133-4eda-8c7b-f7c6aa8a026b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:07.394722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:07.437470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633cf99-d01a-430f-8c67-7f99537461c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:07.438475 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633cf99-d01a-430f-8c67-7f99537461c3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:07.438985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633cf99-d01a-430f-8c67-7f99537461c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:07.439458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633cf99-d01a-430f-8c67-7f99537461c3 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:07.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633cf99-d01a-430f-8c67-7f99537461c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:07.440937 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633cf99-d01a-430f-8c67-7f99537461c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:07.442562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633cf99-d01a-430f-8c67-7f99537461c3 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:07.452411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 360/759] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:07.458133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633cf99-d01a-430f-8c67-7f99537461c3 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:07.458973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4565e-7667-420f-950c-894863246698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:07.459620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4565e-7667-420f-950c-894863246698 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:07.459896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4565e-7667-420f-950c-894863246698 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:07.460192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4565e-7667-420f-950c-894863246698 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:07.460771 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4565e-7667-420f-950c-894863246698 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:07.462659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4565e-7667-420f-950c-894863246698 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-2ed4565e-7667-420f-950c-894863246698 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:07.480260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:07.481322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 11:41:07.482198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:07.485058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 400/760] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Oct 4 11:41:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:07.491420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4565e-7667-420f-950c-894863246698 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:07.560434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:07.676102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633cf99-d01a-430f-8c67-7f99537461c3 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:07.678148 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4565e-7667-420f-950c-894863246698 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:07.715101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacaa7b-55e5-4300-a39d-5979dc45909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:07.716074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacaa7b-55e5-4300-a39d-5979dc45909a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:07.716580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacaa7b-55e5-4300-a39d-5979dc45909a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:07.717070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacaa7b-55e5-4300-a39d-5979dc45909a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:07.729493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacaa7b-55e5-4300-a39d-5979dc45909a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:07.731337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacaa7b-55e5-4300-a39d-5979dc45909a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-ffacaa7b-55e5-4300-a39d-5979dc45909a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:07.747255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:07.748297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:07.749208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:07.749208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:07.749208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:07.751398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 361/761] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:07.757211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacaa7b-55e5-4300-a39d-5979dc45909a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:07.772992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59603a-a020-4056-9cec-1b57428a62c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:07.773748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59603a-a020-4056-9cec-1b57428a62c2 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:07.774033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59603a-a020-4056-9cec-1b57428a62c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:07.774311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59603a-a020-4056-9cec-1b57428a62c2 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:07.775062 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59603a-a020-4056-9cec-1b57428a62c2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:07.775369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59603a-a020-4056-9cec-1b57428a62c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:07.777267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59603a-a020-4056-9cec-1b57428a62c2 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:07.793969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 401/762] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 11:41:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:07.800365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59603a-a020-4056-9cec-1b57428a62c2 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:07.858015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:07.868975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:07.888957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacaa7b-55e5-4300-a39d-5979dc45909a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:07.930882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59603a-a020-4056-9cec-1b57428a62c2 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:07.980496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315797e-ac4d-4bb4-8265-69990a441bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:07.981030 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315797e-ac4d-4bb4-8265-69990a441bf4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:07.981311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315797e-ac4d-4bb4-8265-69990a441bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:07.981585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315797e-ac4d-4bb4-8265-69990a441bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:07.982242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315797e-ac4d-4bb4-8265-69990a441bf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:07.982816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315797e-ac4d-4bb4-8265-69990a441bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:07.985179 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315797e-ac4d-4bb4-8265-69990a441bf4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:08.006592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57afe93-0401-4a74-b6f1-5707affdb711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:08.007276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57afe93-0401-4a74-b6f1-5707affdb711 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:08.007492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57afe93-0401-4a74-b6f1-5707affdb711 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:08.007767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57afe93-0401-4a74-b6f1-5707affdb711 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:08.008472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57afe93-0401-4a74-b6f1-5707affdb711 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:08.008783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57afe93-0401-4a74-b6f1-5707affdb711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:08.012498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57afe93-0401-4a74-b6f1-5707affdb711 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:08.013906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 362/763] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:08.018765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 402/764] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 11:41:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:08.020048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315797e-ac4d-4bb4-8265-69990a441bf4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:08.025467 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57afe93-0401-4a74-b6f1-5707affdb711 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:08.164765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315797e-ac4d-4bb4-8265-69990a441bf4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:08.217210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cf68f-f5cf-46da-8146-a32a9ea19a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:08.217856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cf68f-f5cf-46da-8146-a32a9ea19a52 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:08.218140 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cf68f-f5cf-46da-8146-a32a9ea19a52 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:08.218413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cf68f-f5cf-46da-8146-a32a9ea19a52 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:08.227691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cf68f-f5cf-46da-8146-a32a9ea19a52 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:08.229747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cf68f-f5cf-46da-8146-a32a9ea19a52 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-962cf68f-f5cf-46da-8146-a32a9ea19a52 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:08.244582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:08.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:08.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 363/765] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 11:41:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:08.251307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cf68f-f5cf-46da-8146-a32a9ea19a52 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:08.262622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:08.294530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57afe93-0401-4a74-b6f1-5707affdb711 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:08.303114 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:08.382848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cf68f-f5cf-46da-8146-a32a9ea19a52 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:08.402278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac71d2a-9a05-4731-8d2b-dffc61159f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:08.402901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac71d2a-9a05-4731-8d2b-dffc61159f65 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:08.403180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac71d2a-9a05-4731-8d2b-dffc61159f65 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:08.403451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac71d2a-9a05-4731-8d2b-dffc61159f65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:08.404151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac71d2a-9a05-4731-8d2b-dffc61159f65 None admin] RBAC: Adding query filter params (interface=public, service_id=fb08ee5e7c3c427cbc408f42d340fe83, region_id=RegionOne) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:08.404511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac71d2a-9a05-4731-8d2b-dffc61159f65 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:08.405876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac71d2a-9a05-4731-8d2b-dffc61159f65 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:08.423884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 403/766] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 11:41:08 2022] GET /identity/v3/endpoints?service_id=fb08ee5e7c3c427cbc408f42d340fe83&interface=public®ion_id=RegionOne => generated 194 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:08.425084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740a6d0-7d77-4a8b-b1db-7dbfb9f1b41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:08.426438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740a6d0-7d77-4a8b-b1db-7dbfb9f1b41f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:08.426947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740a6d0-7d77-4a8b-b1db-7dbfb9f1b41f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:08.427441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740a6d0-7d77-4a8b-b1db-7dbfb9f1b41f None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:08.428972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740a6d0-7d77-4a8b-b1db-7dbfb9f1b41f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:08.429476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac71d2a-9a05-4731-8d2b-dffc61159f65 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:08.429758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740a6d0-7d77-4a8b-b1db-7dbfb9f1b41f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:08.431960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740a6d0-7d77-4a8b-b1db-7dbfb9f1b41f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:08.453739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 364/767] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 11:41:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:08.461731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740a6d0-7d77-4a8b-b1db-7dbfb9f1b41f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:08.532558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:08.618047 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740a6d0-7d77-4a8b-b1db-7dbfb9f1b41f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:08.662733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac71d2a-9a05-4731-8d2b-dffc61159f65 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:08.684826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629f9d2-245a-432f-9184-4b9f0c4003c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:08.685618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629f9d2-245a-432f-9184-4b9f0c4003c1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:08.685904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629f9d2-245a-432f-9184-4b9f0c4003c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:08.686254 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629f9d2-245a-432f-9184-4b9f0c4003c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:08.698948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629f9d2-245a-432f-9184-4b9f0c4003c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:08.700437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629f9d2-245a-432f-9184-4b9f0c4003c1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:08.707407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6c254-6df9-4caa-b78e-1e7739f82bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:08.708861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6c254-6df9-4caa-b78e-1e7739f82bdc None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:08.709846 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6c254-6df9-4caa-b78e-1e7739f82bdc None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:08.710338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6c254-6df9-4caa-b78e-1e7739f82bdc None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:08.711440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6c254-6df9-4caa-b78e-1e7739f82bdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:08.713672 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6c254-6df9-4caa-b78e-1e7739f82bdc None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-8629f9d2-245a-432f-9184-4b9f0c4003c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:08.717061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:08.718046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:08.718970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:08.718970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:08.718970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:08.718970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 365/768] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:08.719665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810dfb9-0c22-4af3-9a55-2148ca0ab6be None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:08.719945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810dfb9-0c22-4af3-9a55-2148ca0ab6be None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:08.722539 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 404/769] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 11:41:08 2022] GET /identity/v3/services => generated 772 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:08.732207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810dfb9-0c22-4af3-9a55-2148ca0ab6be None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:08.732663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6c254-6df9-4caa-b78e-1e7739f82bdc None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:08.732780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 366/770] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:08.737946 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5480-fdb6-4641-8876-efeafce3126c None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:08.738470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5480-fdb6-4641-8876-efeafce3126c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:08.738886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5480-fdb6-4641-8876-efeafce3126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:08.768649 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:08.776685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-354e5480-fdb6-4641-8876-efeafce3126c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:08.782926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-354e5480-fdb6-4641-8876-efeafce3126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:08.928126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 367/771] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:08.947735 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6c254-6df9-4caa-b78e-1e7739f82bdc None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:09.000926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc2ac2-3036-4155-8afc-2b991c0c2dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:09.001897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc2ac2-3036-4155-8afc-2b991c0c2dcd None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:09.002350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc2ac2-3036-4155-8afc-2b991c0c2dcd None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:09.002785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc2ac2-3036-4155-8afc-2b991c0c2dcd None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:09.003705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc2ac2-3036-4155-8afc-2b991c0c2dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:09.004006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc2ac2-3036-4155-8afc-2b991c0c2dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:09.006466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc2ac2-3036-4155-8afc-2b991c0c2dcd None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:09.026130 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 405/772] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:09.033636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937a6b9-b879-42e0-8b28-981cdafb7f1c None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:09.033847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc2ac2-3036-4155-8afc-2b991c0c2dcd None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:09.034491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937a6b9-b879-42e0-8b28-981cdafb7f1c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:09.035079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937a6b9-b879-42e0-8b28-981cdafb7f1c None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:09.044325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 368/773] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:09.046155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028a48c-bb34-4243-b5a3-6b7b290fb4a9 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:09.046454 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028a48c-bb34-4243-b5a3-6b7b290fb4a9 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:09.046723 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028a48c-bb34-4243-b5a3-6b7b290fb4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:09.066902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a028a48c-bb34-4243-b5a3-6b7b290fb4a9 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:09.068131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a028a48c-bb34-4243-b5a3-6b7b290fb4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:09.125886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:09.209878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 369/774] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:09.213401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc2ac2-3036-4155-8afc-2b991c0c2dcd None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:09.232902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:09.237978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028a48c-bb34-4243-b5a3-6b7b290fb4a9 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:09.288079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff93325-7cf3-49b7-b716-44d7903dcefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:09.288757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff93325-7cf3-49b7-b716-44d7903dcefe None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:09.289045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff93325-7cf3-49b7-b716-44d7903dcefe None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:09.289315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff93325-7cf3-49b7-b716-44d7903dcefe None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:09.309389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff93325-7cf3-49b7-b716-44d7903dcefe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:09.312135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff93325-7cf3-49b7-b716-44d7903dcefe None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-9ff93325-7cf3-49b7-b716-44d7903dcefe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:09.342306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:09.343286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:09.344210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:09.344210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:09.344210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:09.344210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:09.344210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:09.344210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:09.344210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:09.344210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:09.344210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 406/775] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:09.351279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff93325-7cf3-49b7-b716-44d7903dcefe None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:09.361042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028a48c-bb34-4243-b5a3-6b7b290fb4a9 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:09.409901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ea59a-5c8e-4f57-bc42-05a56a6c6352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:09.410542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ea59a-5c8e-4f57-bc42-05a56a6c6352 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:09.410859 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ea59a-5c8e-4f57-bc42-05a56a6c6352 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:09.411130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ea59a-5c8e-4f57-bc42-05a56a6c6352 None admin] PATH_INFO: `/v3/services/network` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:09.411720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ea59a-5c8e-4f57-bc42-05a56a6c6352 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:09.413144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ea59a-5c8e-4f57-bc42-05a56a6c6352 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-628ea59a-5c8e-4f57-bc42-05a56a6c6352 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:09.437429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:09.440413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 11:41:09.442450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:09.448207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 370/776] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:09 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:09.462271 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ea59a-5c8e-4f57-bc42-05a56a6c6352 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:09.556749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff93325-7cf3-49b7-b716-44d7903dcefe None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:09.560645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:09.600773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793247b0-a29d-4d7e-9223-764e7300a0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:09.601658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793247b0-a29d-4d7e-9223-764e7300a0d6 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:09.601936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793247b0-a29d-4d7e-9223-764e7300a0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:09.602916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793247b0-a29d-4d7e-9223-764e7300a0d6 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:09.603769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793247b0-a29d-4d7e-9223-764e7300a0d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:09.604300 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793247b0-a29d-4d7e-9223-764e7300a0d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:09.607009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793247b0-a29d-4d7e-9223-764e7300a0d6 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:09.632303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 407/777] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:09.633619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793247b0-a29d-4d7e-9223-764e7300a0d6 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:09.663501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ea59a-5c8e-4f57-bc42-05a56a6c6352 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:09.665773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:09.708154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc94c85-fd8d-4e52-9237-446e95ebb7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:09.709550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc94c85-fd8d-4e52-9237-446e95ebb7eb None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:09.709669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc94c85-fd8d-4e52-9237-446e95ebb7eb None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:09.709947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc94c85-fd8d-4e52-9237-446e95ebb7eb None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:09.710975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc94c85-fd8d-4e52-9237-446e95ebb7eb None admin] RBAC: Adding query filter params (name=network) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:09.711454 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc94c85-fd8d-4e52-9237-446e95ebb7eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:09.715006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc94c85-fd8d-4e52-9237-446e95ebb7eb None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:09.724396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 371/778] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:09.725430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc94c85-fd8d-4e52-9237-446e95ebb7eb None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:09.796049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793247b0-a29d-4d7e-9223-764e7300a0d6 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:09.844321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9dbd3-268f-4c35-9a17-d8d05cedf8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:09.845345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9dbd3-268f-4c35-9a17-d8d05cedf8ad None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:09.846093 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9dbd3-268f-4c35-9a17-d8d05cedf8ad None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:09.846592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9dbd3-268f-4c35-9a17-d8d05cedf8ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:09.861238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9dbd3-268f-4c35-9a17-d8d05cedf8ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:09.863870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc94c85-fd8d-4e52-9237-446e95ebb7eb None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-b5b9dbd3-268f-4c35-9a17-d8d05cedf8ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:09.867447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:09.869128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:41:09.870215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:09.870998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:09.871072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 408/779] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:41:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:09.871072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9dbd3-268f-4c35-9a17-d8d05cedf8ad None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:09.908947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e164c-60c2-43c7-be29-924b4e9a08ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:09.909762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e164c-60c2-43c7-be29-924b4e9a08ec None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:09.910093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e164c-60c2-43c7-be29-924b4e9a08ec None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:09.910485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e164c-60c2-43c7-be29-924b4e9a08ec None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/ed14c1bc105d4c5cade5813edc1f0dac/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:09.912502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e164c-60c2-43c7-be29-924b4e9a08ec None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=ed14c1bc105d4c5cade5813edc1f0dac, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:09.913969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e164c-60c2-43c7-be29-924b4e9a08ec None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:09.948772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:09.979837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9dbd3-268f-4c35-9a17-d8d05cedf8ad None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:09.996236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 372/780] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:41:09 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/ed14c1bc105d4c5cade5813edc1f0dac/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:41:09.996236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c0af-fd9c-4106-acd5-7dcf27d484f3 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:09.996236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c0af-fd9c-4106-acd5-7dcf27d484f3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:09.996236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c0af-fd9c-4106-acd5-7dcf27d484f3 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:09.996236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 373/781] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:09.996236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136c0af-fd9c-4106-acd5-7dcf27d484f3 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:10.030801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8b0de-b648-474a-8aae-92c4ba5cdd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:10.031602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8b0de-b648-474a-8aae-92c4ba5cdd9a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:10.031891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8b0de-b648-474a-8aae-92c4ba5cdd9a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:10.032211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8b0de-b648-474a-8aae-92c4ba5cdd9a None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:10.032870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8b0de-b648-474a-8aae-92c4ba5cdd9a None admin] RBAC: Adding query filter params (type=network) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:10.033337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8b0de-b648-474a-8aae-92c4ba5cdd9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:10.034886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8b0de-b648-474a-8aae-92c4ba5cdd9a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:10.042070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 409/782] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:10.043510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfb8ab-bf22-4fd9-aa0c-aa3265cc47c6 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:10.043860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfb8ab-bf22-4fd9-aa0c-aa3265cc47c6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:10.044207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfb8ab-bf22-4fd9-aa0c-aa3265cc47c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:10.065644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-73dfb8ab-bf22-4fd9-aa0c-aa3265cc47c6 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:10.066956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-73dfb8ab-bf22-4fd9-aa0c-aa3265cc47c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:10.147184 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136c0af-fd9c-4106-acd5-7dcf27d484f3 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:10.184435 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 410/783] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:10.185844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfb8ab-bf22-4fd9-aa0c-aa3265cc47c6 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:10.215762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b2d95-5ce5-4916-8040-db65b1d7ab60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:10.216440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2d95-5ce5-4916-8040-db65b1d7ab60 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:10.216797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2d95-5ce5-4916-8040-db65b1d7ab60 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:10.217164 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2d95-5ce5-4916-8040-db65b1d7ab60 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:10.217898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b2d95-5ce5-4916-8040-db65b1d7ab60 None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:10.218299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b2d95-5ce5-4916-8040-db65b1d7ab60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:10.219985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b2d95-5ce5-4916-8040-db65b1d7ab60 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:10.238597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 374/784] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:41:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:10.251074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b2d95-5ce5-4916-8040-db65b1d7ab60 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:10.296588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:10.303916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:10.344021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfb8ab-bf22-4fd9-aa0c-aa3265cc47c6 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:10.411929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1de4fd-f0e0-419e-a39c-bf746cb265a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:10.412961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1de4fd-f0e0-419e-a39c-bf746cb265a4 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:10.413608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1de4fd-f0e0-419e-a39c-bf746cb265a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:10.414074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1de4fd-f0e0-419e-a39c-bf746cb265a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:10.415638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1de4fd-f0e0-419e-a39c-bf746cb265a4 None admin] RBAC: Adding query filter params (interface=public, service_id=4eaf51f54d284cc797c716a56ff44ba7, region_id=RegionOne) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:10.416195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1de4fd-f0e0-419e-a39c-bf746cb265a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:10.418622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1de4fd-f0e0-419e-a39c-bf746cb265a4 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:10.438496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 411/785] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 11:41:10 2022] GET /identity/v3/endpoints?service_id=4eaf51f54d284cc797c716a56ff44ba7&interface=public®ion_id=RegionOne => generated 194 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:10.449243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1de4fd-f0e0-419e-a39c-bf746cb265a4 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:10.497626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b2d95-5ce5-4916-8040-db65b1d7ab60 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:10.547783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941d8ff-0e08-4e33-944b-b44d728f0cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:10.548456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941d8ff-0e08-4e33-944b-b44d728f0cb5 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:10.548737 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941d8ff-0e08-4e33-944b-b44d728f0cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:10.549007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941d8ff-0e08-4e33-944b-b44d728f0cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:10.563261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941d8ff-0e08-4e33-944b-b44d728f0cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:10.565548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941d8ff-0e08-4e33-944b-b44d728f0cb5 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:10.580834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1de4fd-f0e0-419e-a39c-bf746cb265a4 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-8941d8ff-0e08-4e33-944b-b44d728f0cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:10.588932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:10.590067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:10.591564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:10.591564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:10.591564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:10.591564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 375/786] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:10.591564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941d8ff-0e08-4e33-944b-b44d728f0cb5 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:10.623287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:10.630749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80caac3-7fb1-4a2e-bc4b-a289d557ae3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:10.636144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80caac3-7fb1-4a2e-bc4b-a289d557ae3b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:10.636144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80caac3-7fb1-4a2e-bc4b-a289d557ae3b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:10.636144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80caac3-7fb1-4a2e-bc4b-a289d557ae3b None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:10.636144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80caac3-7fb1-4a2e-bc4b-a289d557ae3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:10.636144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80caac3-7fb1-4a2e-bc4b-a289d557ae3b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:10.642012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 412/787] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 11:41:10 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:10.643149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80caac3-7fb1-4a2e-bc4b-a289d557ae3b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:10.738499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:10.775281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80caac3-7fb1-4a2e-bc4b-a289d557ae3b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:10.787723 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941d8ff-0e08-4e33-944b-b44d728f0cb5 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:10.820257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5089a-46d5-4613-ad80-7aff1f88bf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:10.821250 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5089a-46d5-4613-ad80-7aff1f88bf81 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:10.822263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5089a-46d5-4613-ad80-7aff1f88bf81 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:10.822747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5089a-46d5-4613-ad80-7aff1f88bf81 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:10.823683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5089a-46d5-4613-ad80-7aff1f88bf81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:10.824310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5089a-46d5-4613-ad80-7aff1f88bf81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:10.825962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5089a-46d5-4613-ad80-7aff1f88bf81 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:10.840448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 413/788] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:10.846765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5089a-46d5-4613-ad80-7aff1f88bf81 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:10.894878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c6c61-ab29-43eb-aec6-1b2ad30e8a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:10.901069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c6c61-ab29-43eb-aec6-1b2ad30e8a82 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:10.901377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c6c61-ab29-43eb-aec6-1b2ad30e8a82 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:10.901621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c6c61-ab29-43eb-aec6-1b2ad30e8a82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:10.938575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:10.939530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c6c61-ab29-43eb-aec6-1b2ad30e8a82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-560c6c61-ab29-43eb-aec6-1b2ad30e8a82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:10.949070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:10.950097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:10.950982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:10.952196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 376/789] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:10.960752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c6c61-ab29-43eb-aec6-1b2ad30e8a82 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:10.998791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5089a-46d5-4613-ad80-7aff1f88bf81 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:11.039958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219f685-5430-4bf7-85fe-00971ccbd972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:11.042198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219f685-5430-4bf7-85fe-00971ccbd972 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:11.042487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219f685-5430-4bf7-85fe-00971ccbd972 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:11.042762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219f685-5430-4bf7-85fe-00971ccbd972 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:11.051493 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219f685-5430-4bf7-85fe-00971ccbd972 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:11.053105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219f685-5430-4bf7-85fe-00971ccbd972 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-5219f685-5430-4bf7-85fe-00971ccbd972 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:11.066483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:11.067504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:11.068525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 414/790] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 11:41:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:11.073256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219f685-5430-4bf7-85fe-00971ccbd972 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:11.126438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c6c61-ab29-43eb-aec6-1b2ad30e8a82 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:11.193514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219f685-5430-4bf7-85fe-00971ccbd972 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:11.206959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108ad06-a2bc-4496-a22e-8a34c566826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:11.207906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108ad06-a2bc-4496-a22e-8a34c566826f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:11.208410 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108ad06-a2bc-4496-a22e-8a34c566826f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:11.208910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108ad06-a2bc-4496-a22e-8a34c566826f None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:11.209774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108ad06-a2bc-4496-a22e-8a34c566826f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:11.210309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108ad06-a2bc-4496-a22e-8a34c566826f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:11.211885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108ad06-a2bc-4496-a22e-8a34c566826f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:11.224589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:11.227888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 377/791] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:11.229085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:11.234820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108ad06-a2bc-4496-a22e-8a34c566826f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:11.276348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6936f5f-4b94-4f71-ab5e-c8da5afc51ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:11.277091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6936f5f-4b94-4f71-ab5e-c8da5afc51ce None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:11.277370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6936f5f-4b94-4f71-ab5e-c8da5afc51ce None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:11.277641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6936f5f-4b94-4f71-ab5e-c8da5afc51ce None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:11.278365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6936f5f-4b94-4f71-ab5e-c8da5afc51ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:11.278671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6936f5f-4b94-4f71-ab5e-c8da5afc51ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:11.282081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6936f5f-4b94-4f71-ab5e-c8da5afc51ce None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:11.319323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 415/792] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 11:41:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:11.325683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6936f5f-4b94-4f71-ab5e-c8da5afc51ce None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:11.431665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108ad06-a2bc-4496-a22e-8a34c566826f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:11.438206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6936f5f-4b94-4f71-ab5e-c8da5afc51ce None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:11.475007 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e0fc1-7fd5-4dc4-9380-356527e1088d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:11.475583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e0fc1-7fd5-4dc4-9380-356527e1088d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:11.476081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e0fc1-7fd5-4dc4-9380-356527e1088d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:11.476211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e0fc1-7fd5-4dc4-9380-356527e1088d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:11.487359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e0fc1-7fd5-4dc4-9380-356527e1088d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:11.488849 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e0fc1-7fd5-4dc4-9380-356527e1088d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:11.499088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d2a13-f810-43fa-b162-024aaa77c8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:11.499742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d2a13-f810-43fa-b162-024aaa77c8e4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:11.504195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d2a13-f810-43fa-b162-024aaa77c8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:11.504195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d2a13-f810-43fa-b162-024aaa77c8e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-576e0fc1-7fd5-4dc4-9380-356527e1088d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:11.504803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:11.505749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:11.506656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:11.506656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:11.506656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:11.506656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 416/793] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:11.514673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e0fc1-7fd5-4dc4-9380-356527e1088d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:11.533175 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d2a13-f810-43fa-b162-024aaa77c8e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:11.534608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d2a13-f810-43fa-b162-024aaa77c8e4 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:11.544607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-d74d2a13-f810-43fa-b162-024aaa77c8e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:11.574097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:11.575129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:11.576039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:11.576039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:11.576039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:11.576039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 378/794] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:11.597262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d2a13-f810-43fa-b162-024aaa77c8e4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:11.645969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e0fc1-7fd5-4dc4-9380-356527e1088d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:11.715174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f341f9-64ac-45c4-9c4e-e71e2e611471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:11.716283 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f341f9-64ac-45c4-9c4e-e71e2e611471 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:11.716764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f341f9-64ac-45c4-9c4e-e71e2e611471 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:11.717217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f341f9-64ac-45c4-9c4e-e71e2e611471 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:11.718370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f341f9-64ac-45c4-9c4e-e71e2e611471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:11.718889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f341f9-64ac-45c4-9c4e-e71e2e611471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:11.722891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f341f9-64ac-45c4-9c4e-e71e2e611471 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:11.749369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 417/795] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:11.762266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f341f9-64ac-45c4-9c4e-e71e2e611471 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:11.767137 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:11.864901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d2a13-f810-43fa-b162-024aaa77c8e4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:11.911003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143ec51-23a6-417a-b091-5624996d7460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:11.912078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143ec51-23a6-417a-b091-5624996d7460 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:11.912651 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143ec51-23a6-417a-b091-5624996d7460 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:11.913140 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143ec51-23a6-417a-b091-5624996d7460 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:11.914054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143ec51-23a6-417a-b091-5624996d7460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:11.914599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143ec51-23a6-417a-b091-5624996d7460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:11.917094 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143ec51-23a6-417a-b091-5624996d7460 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:11.926005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:11.945865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 379/796] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:11.945865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b729911-0c60-4e90-a4a1-6d7bd23cf56c None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:11.945865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b729911-0c60-4e90-a4a1-6d7bd23cf56c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:11.945865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b729911-0c60-4e90-a4a1-6d7bd23cf56c None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:11.951846 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 380/797] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:11.952991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b729911-0c60-4e90-a4a1-6d7bd23cf56c None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:11.961540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f341f9-64ac-45c4-9c4e-e71e2e611471 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:12.013373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b79fd-52f1-4901-b235-7bfeee9e0e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:12.014400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b79fd-52f1-4901-b235-7bfeee9e0e9b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:12.014918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b79fd-52f1-4901-b235-7bfeee9e0e9b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:12.015397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b79fd-52f1-4901-b235-7bfeee9e0e9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:12.025762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b79fd-52f1-4901-b235-7bfeee9e0e9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:12.027485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b79fd-52f1-4901-b235-7bfeee9e0e9b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-e21b79fd-52f1-4901-b235-7bfeee9e0e9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:12.040331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:12.041768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:12.042666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:12.042666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:12.042666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:12.042666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:12.042666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:12.042666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:12.042666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:12.042666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:12.046403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 418/798] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:12.047867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b2865-2004-43b1-99ed-06c718741a75 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:12.048157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b2865-2004-43b1-99ed-06c718741a75 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:12.048476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b2865-2004-43b1-99ed-06c718741a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:12.069023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-af9b2865-2004-43b1-99ed-06c718741a75 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:12.070037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-af9b2865-2004-43b1-99ed-06c718741a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:12.087747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b729911-0c60-4e90-a4a1-6d7bd23cf56c None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:12.108066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:12.142325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61755682-7ad8-4c47-a04d-f6a7dd51a891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:12.146150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61755682-7ad8-4c47-a04d-f6a7dd51a891 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:12.146806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61755682-7ad8-4c47-a04d-f6a7dd51a891 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:12.147537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61755682-7ad8-4c47-a04d-f6a7dd51a891 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:12.158427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61755682-7ad8-4c47-a04d-f6a7dd51a891 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:12.160481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61755682-7ad8-4c47-a04d-f6a7dd51a891 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:12.160609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 419/799] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:12.161606 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b2865-2004-43b1-99ed-06c718741a75 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-61755682-7ad8-4c47-a04d-f6a7dd51a891 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:12.177483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:12.183441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:12.184453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 381/800] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:41:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:12.190425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61755682-7ad8-4c47-a04d-f6a7dd51a891 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:12.315942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:12.350754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b2865-2004-43b1-99ed-06c718741a75 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:12.383057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61755682-7ad8-4c47-a04d-f6a7dd51a891 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:12.397460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f3335-7134-434f-a88d-50c1a64150df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:12.398481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3335-7134-434f-a88d-50c1a64150df None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:12.398997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3335-7134-434f-a88d-50c1a64150df None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:12.399537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3335-7134-434f-a88d-50c1a64150df None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:12.401083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f3335-7134-434f-a88d-50c1a64150df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:12.401626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f3335-7134-434f-a88d-50c1a64150df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:12.403291 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f3335-7134-434f-a88d-50c1a64150df None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:12.419379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 420/801] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:12.427238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f3335-7134-434f-a88d-50c1a64150df None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:12.460312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f11d82-9f4d-4af1-a0cd-ca7b5632dd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:12.460938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f11d82-9f4d-4af1-a0cd-ca7b5632dd67 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:12.461274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f11d82-9f4d-4af1-a0cd-ca7b5632dd67 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:12.461570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f11d82-9f4d-4af1-a0cd-ca7b5632dd67 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:12.462378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f11d82-9f4d-4af1-a0cd-ca7b5632dd67 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:12.462711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f11d82-9f4d-4af1-a0cd-ca7b5632dd67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:12.464196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f11d82-9f4d-4af1-a0cd-ca7b5632dd67 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:12.499033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 382/802] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 11:41:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:12.506448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f11d82-9f4d-4af1-a0cd-ca7b5632dd67 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:12.572545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:12.645391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f3335-7134-434f-a88d-50c1a64150df None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:12.669272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f11d82-9f4d-4af1-a0cd-ca7b5632dd67 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:12.690914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff052773-f5b5-46d6-91a6-70b450c08a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:12.692163 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff052773-f5b5-46d6-91a6-70b450c08a49 None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:12.692859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff052773-f5b5-46d6-91a6-70b450c08a49 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:12.693381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff052773-f5b5-46d6-91a6-70b450c08a49 None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/952ec01f4a174b6d94c3afd98f90ba81/roles/5b919ba5900449358b3277530a7eff6c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:12.697066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:12.709717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ac00a-a635-4fd3-aea0-dc54d2482dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:12.725608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ac00a-a635-4fd3-aea0-dc54d2482dab None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:12.725608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ac00a-a635-4fd3-aea0-dc54d2482dab None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:12.725608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ac00a-a635-4fd3-aea0-dc54d2482dab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:12.725899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff052773-f5b5-46d6-91a6-70b450c08a49 None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=952ec01f4a174b6d94c3afd98f90ba81, role_id=5b919ba5900449358b3277530a7eff6c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:12.725899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff052773-f5b5-46d6-91a6-70b450c08a49 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:12.726036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ac00a-a635-4fd3-aea0-dc54d2482dab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:12.726036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ac00a-a635-4fd3-aea0-dc54d2482dab None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-c40ac00a-a635-4fd3-aea0-dc54d2482dab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:12.742836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:12.749367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:12.750317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:12.750317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:12.750317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:12.750317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 383/803] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:12.750317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ac00a-a635-4fd3-aea0-dc54d2482dab None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:12.821533 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 421/804] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:41:12 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/952ec01f4a174b6d94c3afd98f90ba81/roles/5b919ba5900449358b3277530a7eff6c => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:41:12.823695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff052773-f5b5-46d6-91a6-70b450c08a49 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:12.871866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:12.894978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ac00a-a635-4fd3-aea0-dc54d2482dab None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:12.941365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13526637-4645-4f9d-a5bc-de78debd0a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:12.941926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13526637-4645-4f9d-a5bc-de78debd0a22 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:12.942205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13526637-4645-4f9d-a5bc-de78debd0a22 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:12.942476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13526637-4645-4f9d-a5bc-de78debd0a22 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:12.943069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13526637-4645-4f9d-a5bc-de78debd0a22 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:12.944524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13526637-4645-4f9d-a5bc-de78debd0a22 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-13526637-4645-4f9d-a5bc-de78debd0a22 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:12.967149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:12.969144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:12.970825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 384/805] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Oct 4 11:41:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:12.978621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13526637-4645-4f9d-a5bc-de78debd0a22 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:13.133705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff052773-f5b5-46d6-91a6-70b450c08a49 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:13.185138 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13526637-4645-4f9d-a5bc-de78debd0a22 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:13.206423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910568a-8c76-49c6-b260-7e66e25dca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:13.207840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910568a-8c76-49c6-b260-7e66e25dca39 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:13.208591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910568a-8c76-49c6-b260-7e66e25dca39 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:13.209274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910568a-8c76-49c6-b260-7e66e25dca39 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:13.210630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910568a-8c76-49c6-b260-7e66e25dca39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:13.211433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910568a-8c76-49c6-b260-7e66e25dca39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:13.214833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910568a-8c76-49c6-b260-7e66e25dca39 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:13.229925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a0069-ed56-4e2c-86cf-9cf77ebaaefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:13.230910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a0069-ed56-4e2c-86cf-9cf77ebaaefc None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:13.231425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a0069-ed56-4e2c-86cf-9cf77ebaaefc None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:13.231906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a0069-ed56-4e2c-86cf-9cf77ebaaefc None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:13.233169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a0069-ed56-4e2c-86cf-9cf77ebaaefc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:13.233800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a0069-ed56-4e2c-86cf-9cf77ebaaefc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:13.235546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a0069-ed56-4e2c-86cf-9cf77ebaaefc None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:13.239156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 422/806] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:13.243493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 385/807] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 11:41:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:13.248204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910568a-8c76-49c6-b260-7e66e25dca39 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:13.249484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a0069-ed56-4e2c-86cf-9cf77ebaaefc None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:13.265144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:13.334057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:13.405502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a0069-ed56-4e2c-86cf-9cf77ebaaefc None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:13.420487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910568a-8c76-49c6-b260-7e66e25dca39 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:13.473222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152ef6e-ca02-4179-8338-e982126400d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:13.474043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152ef6e-ca02-4179-8338-e982126400d5 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:13.474453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152ef6e-ca02-4179-8338-e982126400d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:13.474906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152ef6e-ca02-4179-8338-e982126400d5 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:13.475685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fa54c-41c8-42ef-95ec-516c85bab3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:13.476008 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152ef6e-ca02-4179-8338-e982126400d5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:13.476580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152ef6e-ca02-4179-8338-e982126400d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:13.476744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fa54c-41c8-42ef-95ec-516c85bab3c1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:13.477182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fa54c-41c8-42ef-95ec-516c85bab3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:13.477730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fa54c-41c8-42ef-95ec-516c85bab3c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:13.479096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152ef6e-ca02-4179-8338-e982126400d5 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:13.488428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 386/808] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Oct 4 11:41:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:13.490661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f00b7-e0c8-4260-b8b3-39a39120e571 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:13.491172 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f00b7-e0c8-4260-b8b3-39a39120e571 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:13.491664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f00b7-e0c8-4260-b8b3-39a39120e571 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:13.493964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fa54c-41c8-42ef-95ec-516c85bab3c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:13.494901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 387/809] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:13.496291 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fa54c-41c8-42ef-95ec-516c85bab3c1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:13.498750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411b0ff-c9a2-45c4-b670-3ffcaa7e80db None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:13.498750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411b0ff-c9a2-45c4-b670-3ffcaa7e80db None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:13.498750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411b0ff-c9a2-45c4-b670-3ffcaa7e80db None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:13.500815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 388/810] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:13.502140 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411b0ff-c9a2-45c4-b670-3ffcaa7e80db None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-505fa54c-41c8-42ef-95ec-516c85bab3c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:13.513102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:13.514886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 423/811] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75051ca0-fbc8-48e8-90e4-0c0361416c9d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:13.516774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75051ca0-fbc8-48e8-90e4-0c0361416c9d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:13.517995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75051ca0-fbc8-48e8-90e4-0c0361416c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:13.538731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-75051ca0-fbc8-48e8-90e4-0c0361416c9d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:13.539953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-75051ca0-fbc8-48e8-90e4-0c0361416c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:13.622829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 424/812] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:13.636133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32c2e2-3f46-4d60-abce-5734334ffd6e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:13.637121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32c2e2-3f46-4d60-abce-5734334ffd6e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:13.637809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32c2e2-3f46-4d60-abce-5734334ffd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:13.645244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:13.680913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-fc32c2e2-3f46-4d60-abce-5734334ffd6e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:13.684856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fc32c2e2-3f46-4d60-abce-5734334ffd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:13.692624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411b0ff-c9a2-45c4-b670-3ffcaa7e80db None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:13.754892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:13.758476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00f38d-9df4-44c2-ace8-869dcae43e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:13.759744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f38d-9df4-44c2-ace8-869dcae43e69 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:13.760335 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f38d-9df4-44c2-ace8-869dcae43e69 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:13.760910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f38d-9df4-44c2-ace8-869dcae43e69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:13.762317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00f38d-9df4-44c2-ace8-869dcae43e69 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:13.764900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00f38d-9df4-44c2-ace8-869dcae43e69 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:13.862713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 425/813] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:13.862713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32c2e2-3f46-4d60-abce-5734334ffd6e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:13.891292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 389/814] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 11:41:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:13.896889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00f38d-9df4-44c2-ace8-869dcae43e69 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:14.111873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00f38d-9df4-44c2-ace8-869dcae43e69 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:14.150330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32c2e2-3f46-4d60-abce-5734334ffd6e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:14.208802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:14.221073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d7583-61d8-4d31-b305-5f000e3a3248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:14.221722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d7583-61d8-4d31-b305-5f000e3a3248 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:14.222003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d7583-61d8-4d31-b305-5f000e3a3248 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:14.222281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d7583-61d8-4d31-b305-5f000e3a3248 None admin] PATH_INFO: `/v3/services/network` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:14.222877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d7583-61d8-4d31-b305-5f000e3a3248 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:14.226086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc4e34-b91f-4ad4-b26b-0cb668c0e797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:14.226479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d7583-61d8-4d31-b305-5f000e3a3248 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:14.227729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc4e34-b91f-4ad4-b26b-0cb668c0e797 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:14.229044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc4e34-b91f-4ad4-b26b-0cb668c0e797 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:14.230151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc4e34-b91f-4ad4-b26b-0cb668c0e797 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:14.231899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc4e34-b91f-4ad4-b26b-0cb668c0e797 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:14.232962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc4e34-b91f-4ad4-b26b-0cb668c0e797 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:14.235693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc4e34-b91f-4ad4-b26b-0cb668c0e797 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-5d7d7583-61d8-4d31-b305-5f000e3a3248 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:14.236945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:14.238528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:14.240067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 390/815] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:13 2022] GET /identity/v3/services/network => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:14.245341 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d7583-61d8-4d31-b305-5f000e3a3248 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:14.271717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 426/816] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:14.282138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc4e34-b91f-4ad4-b26b-0cb668c0e797 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:14.357076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:14.435276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d7583-61d8-4d31-b305-5f000e3a3248 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:14.522224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc4e34-b91f-4ad4-b26b-0cb668c0e797 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:14.523005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca05206-635b-4b2b-9bda-3518ca81ad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:14.524276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca05206-635b-4b2b-9bda-3518ca81ad34 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:14.525081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca05206-635b-4b2b-9bda-3518ca81ad34 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:14.526186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca05206-635b-4b2b-9bda-3518ca81ad34 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:14.532908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca05206-635b-4b2b-9bda-3518ca81ad34 None admin] RBAC: Adding query filter params (name=network) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:14.533609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca05206-635b-4b2b-9bda-3518ca81ad34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:14.536603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca05206-635b-4b2b-9bda-3518ca81ad34 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:14.549697 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 391/817] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:14.552743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca05206-635b-4b2b-9bda-3518ca81ad34 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:14.592504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ca933-8c33-4d39-a729-b1f4c8205c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:14.593827 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ca933-8c33-4d39-a729-b1f4c8205c25 None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:14.594456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ca933-8c33-4d39-a729-b1f4c8205c25 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:14.595084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ca933-8c33-4d39-a729-b1f4c8205c25 None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/734d8f03ff2949959cbfce98212a3ac6/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:14.599040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ca933-8c33-4d39-a729-b1f4c8205c25 None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=734d8f03ff2949959cbfce98212a3ac6, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:14.602135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ca933-8c33-4d39-a729-b1f4c8205c25 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:14.669018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:14.686115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 427/818] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:41:14 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/734d8f03ff2949959cbfce98212a3ac6/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:41:14.688219 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ca933-8c33-4d39-a729-b1f4c8205c25 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:14.805462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca05206-635b-4b2b-9bda-3518ca81ad34 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:14.865171 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:14.891520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ca933-8c33-4d39-a729-b1f4c8205c25 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:14.962349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92615dc3-71ee-41c7-a7ee-9a83952c1236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:14.963828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615dc3-71ee-41c7-a7ee-9a83952c1236 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:14.964701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615dc3-71ee-41c7-a7ee-9a83952c1236 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:14.965347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615dc3-71ee-41c7-a7ee-9a83952c1236 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:14.966700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92615dc3-71ee-41c7-a7ee-9a83952c1236 None admin] RBAC: Adding query filter params (type=network) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:14.967453 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92615dc3-71ee-41c7-a7ee-9a83952c1236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:14.970268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92615dc3-71ee-41c7-a7ee-9a83952c1236 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:14.988097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 428/819] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:14.990625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92615dc3-71ee-41c7-a7ee-9a83952c1236 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:15.166113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d125381-f73f-4a8f-b6f0-03f8f8892585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:15.166767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d125381-f73f-4a8f-b6f0-03f8f8892585 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:15.167052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d125381-f73f-4a8f-b6f0-03f8f8892585 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:15.167325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d125381-f73f-4a8f-b6f0-03f8f8892585 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:15.183311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d125381-f73f-4a8f-b6f0-03f8f8892585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-9d125381-f73f-4a8f-b6f0-03f8f8892585 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:15.188915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:15.191067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:15.192965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 392/820] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:41:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 640 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:15.196988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d125381-f73f-4a8f-b6f0-03f8f8892585 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:15.360848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92615dc3-71ee-41c7-a7ee-9a83952c1236 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:15.408633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464738a8-ee7f-469e-b83d-0cc72c6c18d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:15.408948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d125381-f73f-4a8f-b6f0-03f8f8892585 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:15.409320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464738a8-ee7f-469e-b83d-0cc72c6c18d3 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:15.409603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464738a8-ee7f-469e-b83d-0cc72c6c18d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:15.409932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464738a8-ee7f-469e-b83d-0cc72c6c18d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:15.410623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464738a8-ee7f-469e-b83d-0cc72c6c18d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:15.412028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464738a8-ee7f-469e-b83d-0cc72c6c18d3 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:15.440810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:15.480919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be6c09-7d61-48f5-9a3f-7edb63470b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:15.482676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be6c09-7d61-48f5-9a3f-7edb63470b74 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:15.483443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be6c09-7d61-48f5-9a3f-7edb63470b74 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:15.484242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be6c09-7d61-48f5-9a3f-7edb63470b74 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:15.485760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be6c09-7d61-48f5-9a3f-7edb63470b74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:15.486610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be6c09-7d61-48f5-9a3f-7edb63470b74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:15.489577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be6c09-7d61-48f5-9a3f-7edb63470b74 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:15.506652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 393/821] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:41:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:15.519287 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be6c09-7d61-48f5-9a3f-7edb63470b74 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:15.524821 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 429/822] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 11:41:14 2022] POST /identity/v3/endpoints => generated 337 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:15.738736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be6c09-7d61-48f5-9a3f-7edb63470b74 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:15.815166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0683c-c78f-491e-8daa-89389d4d861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:15.816285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0683c-c78f-491e-8daa-89389d4d861e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:15.816765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0683c-c78f-491e-8daa-89389d4d861e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:15.817266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0683c-c78f-491e-8daa-89389d4d861e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:15.837722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0683c-c78f-491e-8daa-89389d4d861e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:15.840494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0683c-c78f-491e-8daa-89389d4d861e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-d6f0683c-c78f-491e-8daa-89389d4d861e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:15.867592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:15.868726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:15.869633 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:15.869633 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:15.869633 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:15.870285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 394/823] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:15.876344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464738a8-ee7f-469e-b83d-0cc72c6c18d3 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:15.884241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:15.954833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0051d-7756-4ced-8dbf-9db0491f3f1e None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:15.955276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0051d-7756-4ced-8dbf-9db0491f3f1e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:15.955758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0051d-7756-4ced-8dbf-9db0491f3f1e None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:15.958592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 395/824] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:15.967626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270f1af-7e6a-4108-96af-41034feaa3b2 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:15.968073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270f1af-7e6a-4108-96af-41034feaa3b2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:15.968576 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270f1af-7e6a-4108-96af-41034feaa3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:16.002744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-8270f1af-7e6a-4108-96af-41034feaa3b2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:16.005036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8270f1af-7e6a-4108-96af-41034feaa3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:16.013523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464738a8-ee7f-469e-b83d-0cc72c6c18d3 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:16.050075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:16.080350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213fc31-7f45-4cda-827f-26b503ceb8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:16.082018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213fc31-7f45-4cda-827f-26b503ceb8d7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:16.082832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213fc31-7f45-4cda-827f-26b503ceb8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:16.083722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213fc31-7f45-4cda-827f-26b503ceb8d7 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:16.085330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213fc31-7f45-4cda-827f-26b503ceb8d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:16.086245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213fc31-7f45-4cda-827f-26b503ceb8d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:16.091635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213fc31-7f45-4cda-827f-26b503ceb8d7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:16.119279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 430/825] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:16.125055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213fc31-7f45-4cda-827f-26b503ceb8d7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:16.163372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 396/826] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:16.205528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:16.251438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213fc31-7f45-4cda-827f-26b503ceb8d7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:16.289900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72611656-0633-4a85-ba04-89db439e1a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:16.291089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72611656-0633-4a85-ba04-89db439e1a10 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:16.291373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72611656-0633-4a85-ba04-89db439e1a10 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:16.291649 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72611656-0633-4a85-ba04-89db439e1a10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:16.300379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72611656-0633-4a85-ba04-89db439e1a10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:16.301838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72611656-0633-4a85-ba04-89db439e1a10 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-72611656-0633-4a85-ba04-89db439e1a10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:16.314462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:16.316280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:16.318745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 431/827] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 11:41:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:16.322319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270f1af-7e6a-4108-96af-41034feaa3b2 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:16.491338 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270f1af-7e6a-4108-96af-41034feaa3b2 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:16.540700 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a069843-5131-4c5e-a64d-a17e52790f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:16.541230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a069843-5131-4c5e-a64d-a17e52790f03 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:16.541511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a069843-5131-4c5e-a64d-a17e52790f03 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:16.541777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a069843-5131-4c5e-a64d-a17e52790f03 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:16.542504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a069843-5131-4c5e-a64d-a17e52790f03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:16.542805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a069843-5131-4c5e-a64d-a17e52790f03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:16.544257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a069843-5131-4c5e-a64d-a17e52790f03 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:16.570352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 397/828] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 11:41:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:16.576068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72611656-0633-4a85-ba04-89db439e1a10 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:16.645682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a069843-5131-4c5e-a64d-a17e52790f03 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:16.667357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:16.688652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72611656-0633-4a85-ba04-89db439e1a10 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:16.721488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:16.737414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7d3c-b497-4198-97b3-90f27e3c75d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:16.738193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7d3c-b497-4198-97b3-90f27e3c75d4 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:16.738706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7d3c-b497-4198-97b3-90f27e3c75d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:16.738949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7d3c-b497-4198-97b3-90f27e3c75d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:16.800922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a069843-5131-4c5e-a64d-a17e52790f03 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:16.827288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c7d3c-b497-4198-97b3-90f27e3c75d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:16.828910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c7d3c-b497-4198-97b3-90f27e3c75d4 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:16.870043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b076c-29f4-4af6-87f2-676122031090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:16.871470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b076c-29f4-4af6-87f2-676122031090 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:16.872210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b076c-29f4-4af6-87f2-676122031090 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:16.872940 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b076c-29f4-4af6-87f2-676122031090 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:16.893411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b076c-29f4-4af6-87f2-676122031090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-035b076c-29f4-4af6-87f2-676122031090 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:16.904247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:16.906424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:41:16.908238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:16.920383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 398/829] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:41:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:16.923719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b076c-29f4-4af6-87f2-676122031090 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:16.952293 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 432/830] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:16 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:41:16.955614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7d3c-b497-4198-97b3-90f27e3c75d4 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:17.006607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:17.033568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:17.095169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b076c-29f4-4af6-87f2-676122031090 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:17.114297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7d3c-b497-4198-97b3-90f27e3c75d4 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:17.156848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39cc22-079e-4b49-a14a-6539a5c1e3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:17.157993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39cc22-079e-4b49-a14a-6539a5c1e3f1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:17.158510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39cc22-079e-4b49-a14a-6539a5c1e3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:17.158921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39cc22-079e-4b49-a14a-6539a5c1e3f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:17.163763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb811d9a-4557-4e2f-90bb-420cd82c7ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:17.165130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb811d9a-4557-4e2f-90bb-420cd82c7ba6 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:17.165813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb811d9a-4557-4e2f-90bb-420cd82c7ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:17.166601 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb811d9a-4557-4e2f-90bb-420cd82c7ba6 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:17.174076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb811d9a-4557-4e2f-90bb-420cd82c7ba6 None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:17.174076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb811d9a-4557-4e2f-90bb-420cd82c7ba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:17.174076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb811d9a-4557-4e2f-90bb-420cd82c7ba6 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:17.174526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39cc22-079e-4b49-a14a-6539a5c1e3f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:17.174526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39cc22-079e-4b49-a14a-6539a5c1e3f1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:17.188736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 399/831] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:41:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-6a39cc22-079e-4b49-a14a-6539a5c1e3f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:17.191441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:17.193164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:17.194950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:17.194950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:17.194950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:17.197090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 433/832] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:17.199145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb811d9a-4557-4e2f-90bb-420cd82c7ba6 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:17.199686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39cc22-079e-4b49-a14a-6539a5c1e3f1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:17.300211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:17.330107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39cc22-079e-4b49-a14a-6539a5c1e3f1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:17.374142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085a22d-bf51-44af-b290-b5843b80133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:17.374397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085a22d-bf51-44af-b290-b5843b80133b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:17.374681 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085a22d-bf51-44af-b290-b5843b80133b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:17.375096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085a22d-bf51-44af-b290-b5843b80133b None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:17.375873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085a22d-bf51-44af-b290-b5843b80133b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:17.376280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085a22d-bf51-44af-b290-b5843b80133b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:17.377850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085a22d-bf51-44af-b290-b5843b80133b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:17.398413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 434/833] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:17.400074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085a22d-bf51-44af-b290-b5843b80133b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:17.407888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb811d9a-4557-4e2f-90bb-420cd82c7ba6 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:17.480762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d76211-2af1-43bb-83b9-d9eea0fa3c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:17.482228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d76211-2af1-43bb-83b9-d9eea0fa3c44 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:17.483290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d76211-2af1-43bb-83b9-d9eea0fa3c44 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:17.484142 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d76211-2af1-43bb-83b9-d9eea0fa3c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:17.487960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:17.504395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d76211-2af1-43bb-83b9-d9eea0fa3c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:17.507076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d76211-2af1-43bb-83b9-d9eea0fa3c44 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:17.519591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085a22d-bf51-44af-b290-b5843b80133b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-70d76211-2af1-43bb-83b9-d9eea0fa3c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:17.533918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:17.534941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:17.535857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:17.535857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:17.535857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:17.536586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 400/834] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:17.538648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feb0c0-1512-41bf-b904-5e5faaf0bea8 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:17.539652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feb0c0-1512-41bf-b904-5e5faaf0bea8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:17.540848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feb0c0-1512-41bf-b904-5e5faaf0bea8 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:17.547796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 401/835] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:17.547796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72feb0c0-1512-41bf-b904-5e5faaf0bea8 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:17.567231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77153c66-aeea-4bbe-a2fc-8857ceec7f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:17.568362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77153c66-aeea-4bbe-a2fc-8857ceec7f9c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:17.568863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77153c66-aeea-4bbe-a2fc-8857ceec7f9c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:17.569349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77153c66-aeea-4bbe-a2fc-8857ceec7f9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:17.575300 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:17.583889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77153c66-aeea-4bbe-a2fc-8857ceec7f9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:17.587968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77153c66-aeea-4bbe-a2fc-8857ceec7f9c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-77153c66-aeea-4bbe-a2fc-8857ceec7f9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:17.610078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:17.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:17.613666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:17.613666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:17.613666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:17.613666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:17.613666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:17.613666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:17.613666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:17.613666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:17.620569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 435/836] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:17.622381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f07db5-e6fe-40bc-955c-e4f309bf3c7c None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:17.622876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f07db5-e6fe-40bc-955c-e4f309bf3c7c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:17.623361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f07db5-e6fe-40bc-955c-e4f309bf3c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:17.651954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-40f07db5-e6fe-40bc-955c-e4f309bf3c7c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:17.653775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-40f07db5-e6fe-40bc-955c-e4f309bf3c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:17.724439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72feb0c0-1512-41bf-b904-5e5faaf0bea8 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:17.756210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 436/837] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:17.756210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f07db5-e6fe-40bc-955c-e4f309bf3c7c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:17.814700 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ac9fe-0ac1-42ef-ae2c-982ebb327385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:17.817021 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac9fe-0ac1-42ef-ae2c-982ebb327385 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:17.818367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac9fe-0ac1-42ef-ae2c-982ebb327385 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:17.819468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ac9fe-0ac1-42ef-ae2c-982ebb327385 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:17.821076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ac9fe-0ac1-42ef-ae2c-982ebb327385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:17.822405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ac9fe-0ac1-42ef-ae2c-982ebb327385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:17.826375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ac9fe-0ac1-42ef-ae2c-982ebb327385 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:17.865054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 402/838] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:17.868929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ac9fe-0ac1-42ef-ae2c-982ebb327385 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:17.964699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:17.966819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f07db5-e6fe-40bc-955c-e4f309bf3c7c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:18.021036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:18.038310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87aba66-76d6-4377-8b72-914f32b25831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:18.039805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aba66-76d6-4377-8b72-914f32b25831 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:18.040716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aba66-76d6-4377-8b72-914f32b25831 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:18.041521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aba66-76d6-4377-8b72-914f32b25831 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:18.043303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87aba66-76d6-4377-8b72-914f32b25831 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:18.044190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87aba66-76d6-4377-8b72-914f32b25831 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:18.047291 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87aba66-76d6-4377-8b72-914f32b25831 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:18.066520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ac9fe-0ac1-42ef-ae2c-982ebb327385 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:18.072474 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 437/839] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:18.074409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87aba66-76d6-4377-8b72-914f32b25831 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:18.110238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8f1be-c6fd-40d2-994e-18731134b457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:18.110877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f1be-c6fd-40d2-994e-18731134b457 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:18.111158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f1be-c6fd-40d2-994e-18731134b457 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:18.111432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f1be-c6fd-40d2-994e-18731134b457 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:18.120499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8f1be-c6fd-40d2-994e-18731134b457 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:18.121999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8f1be-c6fd-40d2-994e-18731134b457 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-c7a8f1be-c6fd-40d2-994e-18731134b457 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:18.135644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:18.143901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 403/840] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 11:41:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:18.145360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8f1be-c6fd-40d2-994e-18731134b457 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:18.251208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87aba66-76d6-4377-8b72-914f32b25831 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:18.285972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8f1be-c6fd-40d2-994e-18731134b457 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:18.293351 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:18.313629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367b033-5989-43e2-9a70-b6259a2db5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:18.314695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367b033-5989-43e2-9a70-b6259a2db5dc None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:18.315175 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367b033-5989-43e2-9a70-b6259a2db5dc None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:18.315675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367b033-5989-43e2-9a70-b6259a2db5dc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:18.316766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367b033-5989-43e2-9a70-b6259a2db5dc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:18.319248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367b033-5989-43e2-9a70-b6259a2db5dc None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:18.322348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817bc83-5137-4ef4-b05a-888983fccc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:18.323065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817bc83-5137-4ef4-b05a-888983fccc37 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:18.323344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817bc83-5137-4ef4-b05a-888983fccc37 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:18.323611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817bc83-5137-4ef4-b05a-888983fccc37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:18.324366 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817bc83-5137-4ef4-b05a-888983fccc37 None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=ed14c1bc105d4c5cade5813edc1f0dac) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:18.324669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817bc83-5137-4ef4-b05a-888983fccc37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:18.326106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817bc83-5137-4ef4-b05a-888983fccc37 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:18.340108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 404/841] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:41:18 2022] GET /identity/v3/role_assignments?user.id=ed14c1bc105d4c5cade5813edc1f0dac&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:18.340108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817bc83-5137-4ef4-b05a-888983fccc37 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-5367b033-5989-43e2-9a70-b6259a2db5dc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:18.340564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:18.341487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:18.342348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 438/842] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:41:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:18.349408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367b033-5989-43e2-9a70-b6259a2db5dc None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:18.438880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:18.518402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367b033-5989-43e2-9a70-b6259a2db5dc None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:18.529402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817bc83-5137-4ef4-b05a-888983fccc37 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:18.564277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31875902-6820-4ada-89ff-f90436e77e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:18.564905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31875902-6820-4ada-89ff-f90436e77e12 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:18.565286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31875902-6820-4ada-89ff-f90436e77e12 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:18.565706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31875902-6820-4ada-89ff-f90436e77e12 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:18.566482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31875902-6820-4ada-89ff-f90436e77e12 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:18.566791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31875902-6820-4ada-89ff-f90436e77e12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:18.568259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31875902-6820-4ada-89ff-f90436e77e12 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:18.573698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 439/843] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:41:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:18.579961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31875902-6820-4ada-89ff-f90436e77e12 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:18.596955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c5ad7-b34c-466e-90b2-8c427d307a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:18.598116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c5ad7-b34c-466e-90b2-8c427d307a5d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:18.598681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c5ad7-b34c-466e-90b2-8c427d307a5d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:18.599099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c5ad7-b34c-466e-90b2-8c427d307a5d None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:18.599896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c5ad7-b34c-466e-90b2-8c427d307a5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:18.600306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c5ad7-b34c-466e-90b2-8c427d307a5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:18.601736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c5ad7-b34c-466e-90b2-8c427d307a5d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:18.626434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 405/844] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 11:41:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:18.632503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c5ad7-b34c-466e-90b2-8c427d307a5d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:18.673555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:18.717098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31875902-6820-4ada-89ff-f90436e77e12 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:18.736193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:18.769647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c5ad7-b34c-466e-90b2-8c427d307a5d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:18.777232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a37f2-5904-4e13-8774-53bde6b0be6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:18.778734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a37f2-5904-4e13-8774-53bde6b0be6a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:18.779424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a37f2-5904-4e13-8774-53bde6b0be6a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:18.780347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a37f2-5904-4e13-8774-53bde6b0be6a None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:18.781800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a37f2-5904-4e13-8774-53bde6b0be6a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:18.783073 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a37f2-5904-4e13-8774-53bde6b0be6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:18.785652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a37f2-5904-4e13-8774-53bde6b0be6a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:18.805025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 440/845] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:41:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:18.806140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6280274-014b-4b5a-98d7-f6c03609657c None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:18.806428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6280274-014b-4b5a-98d7-f6c03609657c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:18.806710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6280274-014b-4b5a-98d7-f6c03609657c None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:18.808586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 441/846] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:18.817743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481516e-dee7-4734-b91a-e64186602d27 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:18.818233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481516e-dee7-4734-b91a-e64186602d27 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:18.818734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481516e-dee7-4734-b91a-e64186602d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:18.832934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35230b-4fbb-465d-aafd-21683e330235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:18.833596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35230b-4fbb-465d-aafd-21683e330235 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:18.833840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35230b-4fbb-465d-aafd-21683e330235 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:18.834115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35230b-4fbb-465d-aafd-21683e330235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:18.844905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-6481516e-dee7-4734-b91a-e64186602d27 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:18.846935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6481516e-dee7-4734-b91a-e64186602d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:18.939133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35230b-4fbb-465d-aafd-21683e330235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:18.941006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35230b-4fbb-465d-aafd-21683e330235 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:18.960644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 442/847] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:18.979450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:19.065413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 406/848] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:41:19.074023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481516e-dee7-4734-b91a-e64186602d27 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:19.191894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481516e-dee7-4734-b91a-e64186602d27 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:19.203547 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:19.230967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de1877-de82-40ce-bf8b-d6a7f49c095d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:19.231612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de1877-de82-40ce-bf8b-d6a7f49c095d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:19.231892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de1877-de82-40ce-bf8b-d6a7f49c095d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:19.232164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de1877-de82-40ce-bf8b-d6a7f49c095d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:19.237004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35230b-4fbb-465d-aafd-21683e330235 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:19.251005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de1877-de82-40ce-bf8b-d6a7f49c095d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:19.251005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de1877-de82-40ce-bf8b-d6a7f49c095d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-11de1877-de82-40ce-bf8b-d6a7f49c095d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:19.268764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:19.269981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:19.271109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:19.271109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:19.271109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:19.272207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 443/849] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:19.281741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de1877-de82-40ce-bf8b-d6a7f49c095d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:19.431252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35230b-4fbb-465d-aafd-21683e330235 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:19.446847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de1877-de82-40ce-bf8b-d6a7f49c095d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:19.466886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:19.477458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee978c-9683-4935-a8d5-ad5d6367fd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:19.488419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee978c-9683-4935-a8d5-ad5d6367fd56 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:19.488419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee978c-9683-4935-a8d5-ad5d6367fd56 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:19.488419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee978c-9683-4935-a8d5-ad5d6367fd56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:19.493852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee978c-9683-4935-a8d5-ad5d6367fd56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-44ee978c-9683-4935-a8d5-ad5d6367fd56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:19.500831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:19.501856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:19.502720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:19.505713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 407/850] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:41:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:19.511461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee978c-9683-4935-a8d5-ad5d6367fd56 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:19.519270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218024a-e6ca-4156-8e6e-fbe453507823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:19.519929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218024a-e6ca-4156-8e6e-fbe453507823 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:19.520468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218024a-e6ca-4156-8e6e-fbe453507823 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:19.520766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218024a-e6ca-4156-8e6e-fbe453507823 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:19.521605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218024a-e6ca-4156-8e6e-fbe453507823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:19.521948 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218024a-e6ca-4156-8e6e-fbe453507823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:19.523589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218024a-e6ca-4156-8e6e-fbe453507823 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:19.544568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 444/851] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:19.547739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218024a-e6ca-4156-8e6e-fbe453507823 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:19.574905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:19.631021 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee978c-9683-4935-a8d5-ad5d6367fd56 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:19.670227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d817c-4b91-4dc3-8d82-5067361063d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:19.670842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d817c-4b91-4dc3-8d82-5067361063d3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:19.671719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d817c-4b91-4dc3-8d82-5067361063d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:19.672012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d817c-4b91-4dc3-8d82-5067361063d3 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:19.672800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d817c-4b91-4dc3-8d82-5067361063d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:19.673103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d817c-4b91-4dc3-8d82-5067361063d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:19.675382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d817c-4b91-4dc3-8d82-5067361063d3 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:19.686784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218024a-e6ca-4156-8e6e-fbe453507823 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:19.688704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 408/852] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:41:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:19.695224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d817c-4b91-4dc3-8d82-5067361063d3 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:19.749133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee72f5d-14a3-4e3a-b470-657570b30c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:19.750601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee72f5d-14a3-4e3a-b470-657570b30c98 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:19.751278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee72f5d-14a3-4e3a-b470-657570b30c98 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:19.751886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee72f5d-14a3-4e3a-b470-657570b30c98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:19.762928 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:19.767757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee72f5d-14a3-4e3a-b470-657570b30c98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:19.771246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee72f5d-14a3-4e3a-b470-657570b30c98 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-cee72f5d-14a3-4e3a-b470-657570b30c98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:19.790689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:19.792808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:19.794671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:19.794671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:19.794671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:19.794671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:19.794671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:19.794671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:19.794671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:19.794671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:19.796895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 445/853] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:19.798356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee72f5d-14a3-4e3a-b470-657570b30c98 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:19.883176 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d817c-4b91-4dc3-8d82-5067361063d3 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:19.924715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563282eb-20f6-4520-a3ca-f931e187e771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:19.925418 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563282eb-20f6-4520-a3ca-f931e187e771 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:19.925711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563282eb-20f6-4520-a3ca-f931e187e771 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:19.925961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563282eb-20f6-4520-a3ca-f931e187e771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:19.930869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:19.939128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563282eb-20f6-4520-a3ca-f931e187e771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:19.940605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563282eb-20f6-4520-a3ca-f931e187e771 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-563282eb-20f6-4520-a3ca-f931e187e771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:19.956653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:19.962910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:19.963826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:19.963826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:19.963826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:19.963826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 409/854] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:19.964057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee72f5d-14a3-4e3a-b470-657570b30c98 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:19.968607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563282eb-20f6-4520-a3ca-f931e187e771 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:20.004490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a871f-3707-40a3-b6a3-ff3b6e0ebca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:20.005369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a871f-3707-40a3-b6a3-ff3b6e0ebca3 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:20.005803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a871f-3707-40a3-b6a3-ff3b6e0ebca3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:20.006226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a871f-3707-40a3-b6a3-ff3b6e0ebca3 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:20.007336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a871f-3707-40a3-b6a3-ff3b6e0ebca3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:20.007826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a871f-3707-40a3-b6a3-ff3b6e0ebca3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:20.010891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a871f-3707-40a3-b6a3-ff3b6e0ebca3 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:20.027221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 446/855] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:20.033222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a871f-3707-40a3-b6a3-ff3b6e0ebca3 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:20.100352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563282eb-20f6-4520-a3ca-f931e187e771 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:20.104671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:20.166153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a871f-3707-40a3-b6a3-ff3b6e0ebca3 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:20.168228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb21b6-5434-46c3-bbce-e2dfb88c96d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:20.169586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb21b6-5434-46c3-bbce-e2dfb88c96d9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:20.170357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb21b6-5434-46c3-bbce-e2dfb88c96d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:20.171222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb21b6-5434-46c3-bbce-e2dfb88c96d9 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:20.172733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb21b6-5434-46c3-bbce-e2dfb88c96d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:20.173552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb21b6-5434-46c3-bbce-e2dfb88c96d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:20.176482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb21b6-5434-46c3-bbce-e2dfb88c96d9 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:20.197458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 410/856] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:20.203229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb21b6-5434-46c3-bbce-e2dfb88c96d9 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:20.218893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dfac6-06c8-42f1-b523-de528adbc17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:20.220457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dfac6-06c8-42f1-b523-de528adbc17a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:20.221406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dfac6-06c8-42f1-b523-de528adbc17a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:20.221900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dfac6-06c8-42f1-b523-de528adbc17a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:20.223660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dfac6-06c8-42f1-b523-de528adbc17a None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=952ec01f4a174b6d94c3afd98f90ba81) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:20.224189 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dfac6-06c8-42f1-b523-de528adbc17a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:20.226832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dfac6-06c8-42f1-b523-de528adbc17a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:20.239227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 447/857] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:41:20 2022] GET /identity/v3/role_assignments?user.id=952ec01f4a174b6d94c3afd98f90ba81&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:20.377191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb21b6-5434-46c3-bbce-e2dfb88c96d9 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:20.435306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e3823-77e1-420c-8902-c7e040fd8df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:20.435949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e3823-77e1-420c-8902-c7e040fd8df1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:20.436261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e3823-77e1-420c-8902-c7e040fd8df1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:20.436529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e3823-77e1-420c-8902-c7e040fd8df1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:20.445002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e3823-77e1-420c-8902-c7e040fd8df1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:20.447355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e3823-77e1-420c-8902-c7e040fd8df1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-ca1e3823-77e1-420c-8902-c7e040fd8df1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:20.460542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:20.461549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:20.463089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 411/858] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:41:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:20.470204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dfac6-06c8-42f1-b523-de528adbc17a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:20.475309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:20.612528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dfac6-06c8-42f1-b523-de528adbc17a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:20.649132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e8b9c-6eae-4ba1-9793-2fd99d4d4e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:20.649807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e8b9c-6eae-4ba1-9793-2fd99d4d4e7e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:20.650080 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e8b9c-6eae-4ba1-9793-2fd99d4d4e7e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:20.650352 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e8b9c-6eae-4ba1-9793-2fd99d4d4e7e None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:20.651119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e8b9c-6eae-4ba1-9793-2fd99d4d4e7e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:20.651424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e8b9c-6eae-4ba1-9793-2fd99d4d4e7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:20.652913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e8b9c-6eae-4ba1-9793-2fd99d4d4e7e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:20.672713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 448/859] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Oct 4 11:41:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:20.682512 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e3823-77e1-420c-8902-c7e040fd8df1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:20.720816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:20.810285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e3823-77e1-420c-8902-c7e040fd8df1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:20.825789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2144e-f5da-43cb-8519-a7cd1b04fdf0 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:20.826751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2144e-f5da-43cb-8519-a7cd1b04fdf0 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:20.827253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2144e-f5da-43cb-8519-a7cd1b04fdf0 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:20.829152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 449/860] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:20.834959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa781-ef87-44bb-8137-693ddf138484 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:20.835250 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa781-ef87-44bb-8137-693ddf138484 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:20.835528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa781-ef87-44bb-8137-693ddf138484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:20.852878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4a1ec-2c95-45e5-8772-7f16186e7821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:20.853510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a1ec-2c95-45e5-8772-7f16186e7821 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:20.853794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a1ec-2c95-45e5-8772-7f16186e7821 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:20.854082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a1ec-2c95-45e5-8772-7f16186e7821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:20.857636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-205aa781-ef87-44bb-8137-693ddf138484 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:20.859922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-205aa781-ef87-44bb-8137-693ddf138484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:20.933696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:20.939060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4a1ec-2c95-45e5-8772-7f16186e7821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:20.941361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4a1ec-2c95-45e5-8772-7f16186e7821 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:20.945859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 450/861] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:20.970275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:21.059045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 412/862] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:20 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:41:21.066215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa781-ef87-44bb-8137-693ddf138484 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:21.200188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa781-ef87-44bb-8137-693ddf138484 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:21.238390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4a1ec-2c95-45e5-8772-7f16186e7821 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:21.265808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4affc-49a1-43a7-9326-045cf7182cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:21.266481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4affc-49a1-43a7-9326-045cf7182cce None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:21.266767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4affc-49a1-43a7-9326-045cf7182cce None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:21.267041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4affc-49a1-43a7-9326-045cf7182cce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:21.279901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4affc-49a1-43a7-9326-045cf7182cce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:21.282298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4affc-49a1-43a7-9326-045cf7182cce None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-50e4affc-49a1-43a7-9326-045cf7182cce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:21.301193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:21.302223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:21.303922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:21.303922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:21.303922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:21.303922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 451/863] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:21.308800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4affc-49a1-43a7-9326-045cf7182cce None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:21.368910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:21.380606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4a1ec-2c95-45e5-8772-7f16186e7821 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:21.410131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:21.426678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75690efa-2369-49b0-8997-c8ffef64c197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:21.427898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75690efa-2369-49b0-8997-c8ffef64c197 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:21.428637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75690efa-2369-49b0-8997-c8ffef64c197 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:21.429245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75690efa-2369-49b0-8997-c8ffef64c197 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:21.430043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75690efa-2369-49b0-8997-c8ffef64c197 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:21.432401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75690efa-2369-49b0-8997-c8ffef64c197 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:21.437162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4affc-49a1-43a7-9326-045cf7182cce None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-75690efa-2369-49b0-8997-c8ffef64c197 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:21.446952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:21.448122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 11:41:21.449070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:21.456875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 413/864] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 4 11:41:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:21.458591 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75690efa-2369-49b0-8997-c8ffef64c197 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:21.478624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae21d5-594f-4590-a169-5a57454d2e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:21.479240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae21d5-594f-4590-a169-5a57454d2e93 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:21.479526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae21d5-594f-4590-a169-5a57454d2e93 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:21.479808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae21d5-594f-4590-a169-5a57454d2e93 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:21.480559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae21d5-594f-4590-a169-5a57454d2e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:21.480869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae21d5-594f-4590-a169-5a57454d2e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:21.483247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae21d5-594f-4590-a169-5a57454d2e93 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:21.496785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 452/865] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:21.498132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0e257-9e54-4f2e-8b26-db95a6c19441 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:21.498417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0e257-9e54-4f2e-8b26-db95a6c19441 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:21.498691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0e257-9e54-4f2e-8b26-db95a6c19441 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:21.500089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 453/866] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:21.505039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0e257-9e54-4f2e-8b26-db95a6c19441 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:21.580529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75690efa-2369-49b0-8997-c8ffef64c197 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:21.627076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0e257-9e54-4f2e-8b26-db95a6c19441 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:21.631161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:21.648218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28772e37-3aed-457a-8049-2af0fd140313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:21.648218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28772e37-3aed-457a-8049-2af0fd140313 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:21.653535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28772e37-3aed-457a-8049-2af0fd140313 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:21.653535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28772e37-3aed-457a-8049-2af0fd140313 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:21.653535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28772e37-3aed-457a-8049-2af0fd140313 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:21.653535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28772e37-3aed-457a-8049-2af0fd140313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:21.653535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28772e37-3aed-457a-8049-2af0fd140313 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:21.660544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 414/867] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:21.662212 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90f191-3bd6-4c08-a7cc-8771632bc0a1 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:21.662497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90f191-3bd6-4c08-a7cc-8771632bc0a1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:21.662837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90f191-3bd6-4c08-a7cc-8771632bc0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:21.676324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e4cd6-46b5-4bbb-950b-1649a5e6284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:21.677004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e4cd6-46b5-4bbb-950b-1649a5e6284c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:21.677344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e4cd6-46b5-4bbb-950b-1649a5e6284c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:21.677626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e4cd6-46b5-4bbb-950b-1649a5e6284c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:21.686125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e4cd6-46b5-4bbb-950b-1649a5e6284c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:21.687599 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e4cd6-46b5-4bbb-950b-1649a5e6284c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:21.691378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-ba90f191-3bd6-4c08-a7cc-8771632bc0a1 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:21.693751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ba90f191-3bd6-4c08-a7cc-8771632bc0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-d02e4cd6-46b5-4bbb-950b-1649a5e6284c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:21.700857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:21.702361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:21.703897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:21.703897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:21.703897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:21.703897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:21.703897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:21.703897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:21.703897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:21.703897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:21.703897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 454/868] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:21.703897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e4cd6-46b5-4bbb-950b-1649a5e6284c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:21.767450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:21.811517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 415/869] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:21.811517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90f191-3bd6-4c08-a7cc-8771632bc0a1 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:21.855214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e4cd6-46b5-4bbb-950b-1649a5e6284c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:21.915948 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fdd7a-2812-4a14-9697-d3431eab604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:21.916958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fdd7a-2812-4a14-9697-d3431eab604d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:21.917463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fdd7a-2812-4a14-9697-d3431eab604d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:21.917948 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fdd7a-2812-4a14-9697-d3431eab604d None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:21.918989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fdd7a-2812-4a14-9697-d3431eab604d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:21.919480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fdd7a-2812-4a14-9697-d3431eab604d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:21.921893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fdd7a-2812-4a14-9697-d3431eab604d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:21.932139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 455/870] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:21 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:21.976956 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90f191-3bd6-4c08-a7cc-8771632bc0a1 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:22.020844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffba617-c1fd-4ccc-996e-367dee21f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:22.021564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffba617-c1fd-4ccc-996e-367dee21f234 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:22.021847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffba617-c1fd-4ccc-996e-367dee21f234 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:22.022118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffba617-c1fd-4ccc-996e-367dee21f234 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:22.022830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffba617-c1fd-4ccc-996e-367dee21f234 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:22.023137 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffba617-c1fd-4ccc-996e-367dee21f234 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:22.024742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffba617-c1fd-4ccc-996e-367dee21f234 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:22.040692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 416/871] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:22.049328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fdd7a-2812-4a14-9697-d3431eab604d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:22.082447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:22.161293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffba617-c1fd-4ccc-996e-367dee21f234 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:22.164796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fdd7a-2812-4a14-9697-d3431eab604d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:22.207349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244423f6-8160-4569-ac8b-406e943b8305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:22.208275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:22.208718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244423f6-8160-4569-ac8b-406e943b8305 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:22.209181 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244423f6-8160-4569-ac8b-406e943b8305 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:22.209312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244423f6-8160-4569-ac8b-406e943b8305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:22.209986 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244423f6-8160-4569-ac8b-406e943b8305 None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=734d8f03ff2949959cbfce98212a3ac6) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:22.210321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244423f6-8160-4569-ac8b-406e943b8305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:22.211850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244423f6-8160-4569-ac8b-406e943b8305 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:22.222879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 456/872] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:41:22 2022] GET /identity/v3/role_assignments?user.id=734d8f03ff2949959cbfce98212a3ac6&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:22.290429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffba617-c1fd-4ccc-996e-367dee21f234 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:22.332195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c6ce4-2e77-4a72-9842-d3266d414b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:22.332919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c6ce4-2e77-4a72-9842-d3266d414b7d None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:22.333201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c6ce4-2e77-4a72-9842-d3266d414b7d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:22.333476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c6ce4-2e77-4a72-9842-d3266d414b7d None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:22.334164 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c6ce4-2e77-4a72-9842-d3266d414b7d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:22.335626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c6ce4-2e77-4a72-9842-d3266d414b7d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:22.461363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 417/873] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:41:22 2022] POST /identity/v3/services => generated 251 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:22.655998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009675a-1e40-4f42-972a-23142bd70c83 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:22.656512 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009675a-1e40-4f42-972a-23142bd70c83 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:22.656896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009675a-1e40-4f42-972a-23142bd70c83 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:22.658818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 457/874] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:22.664904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b6a5d-dc5b-4190-816f-ae5b539c46e5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:22.665337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b6a5d-dc5b-4190-816f-ae5b539c46e5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:22.665716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b6a5d-dc5b-4190-816f-ae5b539c46e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:22.686112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-228b6a5d-dc5b-4190-816f-ae5b539c46e5 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:22.687404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-228b6a5d-dc5b-4190-816f-ae5b539c46e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:22.781429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 418/875] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:22 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:23.051798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009675a-1e40-4f42-972a-23142bd70c83 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:23.180725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:23.191109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009675a-1e40-4f42-972a-23142bd70c83 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:23.229563 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e4dfe-a6e1-41b2-a662-87e402aef3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:23.230212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e4dfe-a6e1-41b2-a662-87e402aef3b4 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:23.230492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e4dfe-a6e1-41b2-a662-87e402aef3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:23.230764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e4dfe-a6e1-41b2-a662-87e402aef3b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:23.243274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e4dfe-a6e1-41b2-a662-87e402aef3b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-120e4dfe-a6e1-41b2-a662-87e402aef3b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:23.248222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:23.249162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:23.250009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 458/876] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 11:41:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:23.257264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b6a5d-dc5b-4190-816f-ae5b539c46e5 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:23.290948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:23.397759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b6a5d-dc5b-4190-816f-ae5b539c46e5 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:23.448252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74b232-3643-4f94-b9a5-3be4bc0c459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:23.449712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74b232-3643-4f94-b9a5-3be4bc0c459c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:23.450431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74b232-3643-4f94-b9a5-3be4bc0c459c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:23.451161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74b232-3643-4f94-b9a5-3be4bc0c459c None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:23.452719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74b232-3643-4f94-b9a5-3be4bc0c459c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:23.453587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74b232-3643-4f94-b9a5-3be4bc0c459c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:23.456431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74b232-3643-4f94-b9a5-3be4bc0c459c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:23.469073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 419/877] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 11:41:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:23.477967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e4dfe-a6e1-41b2-a662-87e402aef3b4 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:23.610566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e4dfe-a6e1-41b2-a662-87e402aef3b4 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:23.662533 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313ab85-1743-460e-867c-fa6550d9233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:23.663686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313ab85-1743-460e-867c-fa6550d9233e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:23.664361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313ab85-1743-460e-867c-fa6550d9233e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:23.665625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313ab85-1743-460e-867c-fa6550d9233e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:23.682551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c313ab85-1743-460e-867c-fa6550d9233e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:23.684871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c313ab85-1743-460e-867c-fa6550d9233e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-c313ab85-1743-460e-867c-fa6550d9233e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:23.710825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:23.712113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:23.713413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:23.713413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:23.713413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:23.715766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 459/878] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:23.723640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74b232-3643-4f94-b9a5-3be4bc0c459c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:23.852609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:23.856758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74b232-3643-4f94-b9a5-3be4bc0c459c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:23.893725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b4113-a891-4a78-93fc-b97a18de02d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:23.894344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b4113-a891-4a78-93fc-b97a18de02d5 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:23.894624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b4113-a891-4a78-93fc-b97a18de02d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:23.894896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b4113-a891-4a78-93fc-b97a18de02d5 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:23.895549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b4113-a891-4a78-93fc-b97a18de02d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:23.895833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b4113-a891-4a78-93fc-b97a18de02d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:23.897243 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b4113-a891-4a78-93fc-b97a18de02d5 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:23.910752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 420/879] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:23.917212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313ab85-1743-460e-867c-fa6550d9233e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:23.921090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:24.031409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313ab85-1743-460e-867c-fa6550d9233e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:24.066783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961c4ab-3b49-4c58-8e13-fe4ceb58ca79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:24.068292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961c4ab-3b49-4c58-8e13-fe4ceb58ca79 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:24.068811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961c4ab-3b49-4c58-8e13-fe4ceb58ca79 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:24.069285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961c4ab-3b49-4c58-8e13-fe4ceb58ca79 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:24.078302 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961c4ab-3b49-4c58-8e13-fe4ceb58ca79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:24.080037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961c4ab-3b49-4c58-8e13-fe4ceb58ca79 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-5961c4ab-3b49-4c58-8e13-fe4ceb58ca79 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:24.091936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:24.093103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:24.094065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:24.094065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:24.094065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:24.094065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:24.094065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:24.094065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:24.094065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:24.094065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:24.094065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:24.094065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:24.096604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 460/880] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 11:41:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:24.105465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b4113-a891-4a78-93fc-b97a18de02d5 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:24.272096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1afa0-c577-41b0-9412-92c0556ee2aa None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:24.272465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1afa0-c577-41b0-9412-92c0556ee2aa None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:24.272643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1afa0-c577-41b0-9412-92c0556ee2aa None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:24.274805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 461/881] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:24.280125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634358fc-4fc4-4af8-805d-81b8d47d2e6f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:24.280458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634358fc-4fc4-4af8-805d-81b8d47d2e6f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:24.280735 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634358fc-4fc4-4af8-805d-81b8d47d2e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:24.300706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b4113-a891-4a78-93fc-b97a18de02d5 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:24.302024 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-634358fc-4fc4-4af8-805d-81b8d47d2e6f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:24.303317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-634358fc-4fc4-4af8-805d-81b8d47d2e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:24.367791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:24.370492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09284075-5c30-466d-871e-de48e0864bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:24.371612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09284075-5c30-466d-871e-de48e0864bc7 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:24.372117 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09284075-5c30-466d-871e-de48e0864bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:24.372699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09284075-5c30-466d-871e-de48e0864bc7 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:24.374071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09284075-5c30-466d-871e-de48e0864bc7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:24.374818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09284075-5c30-466d-871e-de48e0864bc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:24.384222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09284075-5c30-466d-871e-de48e0864bc7 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:24.384945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 462/882] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:24 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:24.420617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 421/883] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 11:41:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:24.425471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634358fc-4fc4-4af8-805d-81b8d47d2e6f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:24.596915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634358fc-4fc4-4af8-805d-81b8d47d2e6f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:24.634642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721419e-2262-419b-a390-426a36dca80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:24.635258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721419e-2262-419b-a390-426a36dca80b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:24.635531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721419e-2262-419b-a390-426a36dca80b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:24.635786 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721419e-2262-419b-a390-426a36dca80b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:24.658855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09284075-5c30-466d-871e-de48e0864bc7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:24.668695 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:24.720989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:24.730088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721419e-2262-419b-a390-426a36dca80b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:24.731576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721419e-2262-419b-a390-426a36dca80b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:24.823563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09284075-5c30-466d-871e-de48e0864bc7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:24.854186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 463/884] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:24 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 11:41:24.855686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b864b-5557-4181-b928-fa1697d80920 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:24.856507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b864b-5557-4181-b928-fa1697d80920 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:24.856507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b864b-5557-4181-b928-fa1697d80920 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:24.862052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 464/885] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:24.863212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b864b-5557-4181-b928-fa1697d80920 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:24.866438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6242409-0b1f-426b-82a1-b2780eeaa9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:24.867046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6242409-0b1f-426b-82a1-b2780eeaa9a1 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:24.867549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6242409-0b1f-426b-82a1-b2780eeaa9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:24.868274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6242409-0b1f-426b-82a1-b2780eeaa9a1 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:24.868804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6242409-0b1f-426b-82a1-b2780eeaa9a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:24.870186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6242409-0b1f-426b-82a1-b2780eeaa9a1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:24.944406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 422/886] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:41:24 2022] POST /identity/v3/services => generated 269 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:41:24.947490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b1c36-4d6d-4787-95ba-93bbfd5bb9d3 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:24.947781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b1c36-4d6d-4787-95ba-93bbfd5bb9d3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:24.948384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b1c36-4d6d-4787-95ba-93bbfd5bb9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:24.972352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-ba8b1c36-4d6d-4787-95ba-93bbfd5bb9d3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:24.975390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ba8b1c36-4d6d-4787-95ba-93bbfd5bb9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:25.029744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:25.044648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b864b-5557-4181-b928-fa1697d80920 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:25.075123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:25.091219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 423/887] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:25.091219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5ac70-9274-49e7-8d6d-bf36e9224ed5 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:25.091219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5ac70-9274-49e7-8d6d-bf36e9224ed5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:25.091219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5ac70-9274-49e7-8d6d-bf36e9224ed5 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:25.091219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 424/888] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:25.099506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746802-560e-4a86-9309-bdebebeb33c3 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:25.099771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746802-560e-4a86-9309-bdebebeb33c3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:25.100050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746802-560e-4a86-9309-bdebebeb33c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:25.107800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510feaa0-141b-4d86-a252-1a7498c3a388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:25.111143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510feaa0-141b-4d86-a252-1a7498c3a388 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:25.111883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510feaa0-141b-4d86-a252-1a7498c3a388 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:25.112582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510feaa0-141b-4d86-a252-1a7498c3a388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:25.122436 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-33746802-560e-4a86-9309-bdebebeb33c3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:25.123818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-33746802-560e-4a86-9309-bdebebeb33c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:25.139028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510feaa0-141b-4d86-a252-1a7498c3a388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:25.142869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510feaa0-141b-4d86-a252-1a7498c3a388 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-510feaa0-141b-4d86-a252-1a7498c3a388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:25.175256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:25.176335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:25.177412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:25.177412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:25.177412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:25.177694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 465/889] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:25.186212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510feaa0-141b-4d86-a252-1a7498c3a388 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:25.226669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 425/890] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:25.321742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510feaa0-141b-4d86-a252-1a7498c3a388 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:25.360935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34561870-ba66-4a18-81e0-51951743bc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:25.361489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34561870-ba66-4a18-81e0-51951743bc14 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:25.361776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34561870-ba66-4a18-81e0-51951743bc14 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:25.362053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34561870-ba66-4a18-81e0-51951743bc14 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:25.362704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34561870-ba66-4a18-81e0-51951743bc14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:25.363010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34561870-ba66-4a18-81e0-51951743bc14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:25.364416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34561870-ba66-4a18-81e0-51951743bc14 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:25.378661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 466/891] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:25.387664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33746802-560e-4a86-9309-bdebebeb33c3 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:25.507671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34561870-ba66-4a18-81e0-51951743bc14 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:25.511402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:25.511673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33746802-560e-4a86-9309-bdebebeb33c3 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:25.540374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:25.549113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d12f33-24ee-4a99-a625-6c81f26bfaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:25.549762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d12f33-24ee-4a99-a625-6c81f26bfaea None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:25.550037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d12f33-24ee-4a99-a625-6c81f26bfaea None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:25.550299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d12f33-24ee-4a99-a625-6c81f26bfaea None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:25.558448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d12f33-24ee-4a99-a625-6c81f26bfaea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:25.559918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d12f33-24ee-4a99-a625-6c81f26bfaea None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-c4d12f33-24ee-4a99-a625-6c81f26bfaea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:25.572735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:25.573732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:25.574798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:25.574798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:25.574798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:25.574798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:25.574798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:25.574798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:25.574798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:25.574798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:25.574798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 426/892] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:25.583494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d12f33-24ee-4a99-a625-6c81f26bfaea None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:25.654465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34561870-ba66-4a18-81e0-51951743bc14 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:25.707627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d12f33-24ee-4a99-a625-6c81f26bfaea None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:25.720562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed8b2e-cfa7-4092-8eef-bd9b90e2ddbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:25.721619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed8b2e-cfa7-4092-8eef-bd9b90e2ddbc None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:25.722133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed8b2e-cfa7-4092-8eef-bd9b90e2ddbc None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:25.722633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed8b2e-cfa7-4092-8eef-bd9b90e2ddbc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:25.723643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed8b2e-cfa7-4092-8eef-bd9b90e2ddbc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:25.726309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed8b2e-cfa7-4092-8eef-bd9b90e2ddbc None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-56ed8b2e-cfa7-4092-8eef-bd9b90e2ddbc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:25.745033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:25.747094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:25.748901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 467/893] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:25.750168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed8b2e-cfa7-4092-8eef-bd9b90e2ddbc None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:25.755629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb291bc2-fb66-4075-84f4-78bbcb6f5700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:25.755629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb291bc2-fb66-4075-84f4-78bbcb6f5700 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:25.756155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb291bc2-fb66-4075-84f4-78bbcb6f5700 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:25.756155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb291bc2-fb66-4075-84f4-78bbcb6f5700 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:25.756706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb291bc2-fb66-4075-84f4-78bbcb6f5700 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:25.757010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb291bc2-fb66-4075-84f4-78bbcb6f5700 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:25.758526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb291bc2-fb66-4075-84f4-78bbcb6f5700 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:25.771695 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 427/894] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:25.772614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb291bc2-fb66-4075-84f4-78bbcb6f5700 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:25.838726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:25.866698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:25.896324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb291bc2-fb66-4075-84f4-78bbcb6f5700 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:25.921949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed8b2e-cfa7-4092-8eef-bd9b90e2ddbc None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:25.953296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2a2fa-69f7-4e31-89a0-75b36f55f27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:25.954384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2a2fa-69f7-4e31-89a0-75b36f55f27f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:25.954889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2a2fa-69f7-4e31-89a0-75b36f55f27f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:25.955380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2a2fa-69f7-4e31-89a0-75b36f55f27f None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:25.956468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2a2fa-69f7-4e31-89a0-75b36f55f27f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:25.956984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2a2fa-69f7-4e31-89a0-75b36f55f27f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:25.957322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbca5b-c90e-4064-bfb4-8d01247341e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:25.957946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbca5b-c90e-4064-bfb4-8d01247341e8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:25.958223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbca5b-c90e-4064-bfb4-8d01247341e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:25.958495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbca5b-c90e-4064-bfb4-8d01247341e8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:25.959074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbca5b-c90e-4064-bfb4-8d01247341e8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:25.959426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2a2fa-69f7-4e31-89a0-75b36f55f27f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:25.960439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbca5b-c90e-4064-bfb4-8d01247341e8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:25.966886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 428/895] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 11:41:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:25.967698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2a2fa-69f7-4e31-89a0-75b36f55f27f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-44cbca5b-c90e-4064-bfb4-8d01247341e8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:25.973648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:25.974653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:25.976305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 468/896] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:41:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:25.978381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbca5b-c90e-4064-bfb4-8d01247341e8 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:26.106748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2a2fa-69f7-4e31-89a0-75b36f55f27f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:26.118870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:26.130524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbca5b-c90e-4064-bfb4-8d01247341e8 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:26.146071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbae8c9-e123-4aad-af27-9ea34830f3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:26.146789 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbae8c9-e123-4aad-af27-9ea34830f3e3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:26.147068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbae8c9-e123-4aad-af27-9ea34830f3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:26.147336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbae8c9-e123-4aad-af27-9ea34830f3e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:26.148393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbae8c9-e123-4aad-af27-9ea34830f3e3 None admin] RBAC: Adding query filter params (role.id=87a27c6c32d94ae7b373df89e15cb25e, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=952ec01f4a174b6d94c3afd98f90ba81) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:26.148690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbae8c9-e123-4aad-af27-9ea34830f3e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:26.150932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbae8c9-e123-4aad-af27-9ea34830f3e3 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:26.165866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 429/897] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:41:25 2022] GET /identity/v3/role_assignments?user.id=952ec01f4a174b6d94c3afd98f90ba81&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=87a27c6c32d94ae7b373df89e15cb25e => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:26.166685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbae8c9-e123-4aad-af27-9ea34830f3e3 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:26.188914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:26.199752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12abd9-aa09-4439-bea8-79f8ff7b6660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:26.205634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12abd9-aa09-4439-bea8-79f8ff7b6660 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:26.205634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12abd9-aa09-4439-bea8-79f8ff7b6660 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:26.205634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12abd9-aa09-4439-bea8-79f8ff7b6660 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:26.205634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12abd9-aa09-4439-bea8-79f8ff7b6660 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:26.205634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12abd9-aa09-4439-bea8-79f8ff7b6660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:26.206286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12abd9-aa09-4439-bea8-79f8ff7b6660 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:26.214704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 469/898] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 11:41:25 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:26.360427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbae8c9-e123-4aad-af27-9ea34830f3e3 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:26.415689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b6b3d-d113-463b-92aa-39cb64eaf4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:26.416642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b6b3d-d113-463b-92aa-39cb64eaf4b6 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:26.416805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b6b3d-d113-463b-92aa-39cb64eaf4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:26.416926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b6b3d-d113-463b-92aa-39cb64eaf4b6 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:26.417672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b6b3d-d113-463b-92aa-39cb64eaf4b6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:26.417990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b6b3d-d113-463b-92aa-39cb64eaf4b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:26.419456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b6b3d-d113-463b-92aa-39cb64eaf4b6 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:26.425452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 430/899] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:41:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:26.432546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12abd9-aa09-4439-bea8-79f8ff7b6660 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:26.615225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12abd9-aa09-4439-bea8-79f8ff7b6660 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:26.679548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d661b0-f36b-4766-9bfc-813078cd2531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:26.681148 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d661b0-f36b-4766-9bfc-813078cd2531 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:26.681640 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d661b0-f36b-4766-9bfc-813078cd2531 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:26.682132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d661b0-f36b-4766-9bfc-813078cd2531 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:26.683371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d661b0-f36b-4766-9bfc-813078cd2531 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:26.683935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d661b0-f36b-4766-9bfc-813078cd2531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:26.686672 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d661b0-f36b-4766-9bfc-813078cd2531 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:26.695057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 470/900] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:41:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:26.701406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b6b3d-d113-463b-92aa-39cb64eaf4b6 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:26.747550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:26.852464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b6b3d-d113-463b-92aa-39cb64eaf4b6 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:26.914572 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6f9de-01ee-4cde-a81b-5a774debacec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:26.915565 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6f9de-01ee-4cde-a81b-5a774debacec None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:26.916107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6f9de-01ee-4cde-a81b-5a774debacec None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:26.916549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6f9de-01ee-4cde-a81b-5a774debacec None admin] PATH_INFO: `/v3/endpoints` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:26.917419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6f9de-01ee-4cde-a81b-5a774debacec None admin] RBAC: Adding query filter params (interface=public, service_id=c9f97f4eaa164ac58561c2ed1b59050c, region_id=RegionOne) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:26.917908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6f9de-01ee-4cde-a81b-5a774debacec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:26.919478 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a6f9de-01ee-4cde-a81b-5a774debacec None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:26.928710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 431/901] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 11:41:26 2022] GET /identity/v3/endpoints?service_id=c9f97f4eaa164ac58561c2ed1b59050c&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:26.940804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d661b0-f36b-4766-9bfc-813078cd2531 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:27.021329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:27.073916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d661b0-f36b-4766-9bfc-813078cd2531 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:27.116605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51585ae4-8117-485b-b2d2-65c3b8d5e627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:27.117465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51585ae4-8117-485b-b2d2-65c3b8d5e627 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:27.118266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51585ae4-8117-485b-b2d2-65c3b8d5e627 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:27.118574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51585ae4-8117-485b-b2d2-65c3b8d5e627 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:27.119182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51585ae4-8117-485b-b2d2-65c3b8d5e627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:27.120726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51585ae4-8117-485b-b2d2-65c3b8d5e627 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:27.128059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 471/902] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 11:41:26 2022] GET /identity/v3/services => generated 1268 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 11:41:27.864913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12a283-79ea-4f11-bbde-079bd55ab897 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:27.865497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12a283-79ea-4f11-bbde-079bd55ab897 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:27.865855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12a283-79ea-4f11-bbde-079bd55ab897 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:27.869209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 432/903] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:27.877603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075300d-f608-4eb6-a43a-df697270e8d3 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:27.878107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075300d-f608-4eb6-a43a-df697270e8d3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:27.878554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075300d-f608-4eb6-a43a-df697270e8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:27.908881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-8075300d-f608-4eb6-a43a-df697270e8d3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:27.910890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8075300d-f608-4eb6-a43a-df697270e8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:28.027714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 472/904] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:28.298880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12a283-79ea-4f11-bbde-079bd55ab897 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:28.432882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12a283-79ea-4f11-bbde-079bd55ab897 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:28.434824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:28.480931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85c5f0-6a7c-422c-beee-21f55d7aa719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:28.482339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85c5f0-6a7c-422c-beee-21f55d7aa719 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:28.483301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85c5f0-6a7c-422c-beee-21f55d7aa719 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:28.483975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85c5f0-6a7c-422c-beee-21f55d7aa719 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:28.485246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85c5f0-6a7c-422c-beee-21f55d7aa719 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:28.487991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85c5f0-6a7c-422c-beee-21f55d7aa719 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-ab85c5f0-6a7c-422c-beee-21f55d7aa719 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:28.506066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:28.508303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 11:41:28.510329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:28.514687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 433/905] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 4 11:41:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:28.521173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075300d-f608-4eb6-a43a-df697270e8d3 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:28.621305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:28.684038 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075300d-f608-4eb6-a43a-df697270e8d3 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:28.722281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814047b2-223f-40e1-b018-0c5eea563be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:28.723168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814047b2-223f-40e1-b018-0c5eea563be1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:28.723699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814047b2-223f-40e1-b018-0c5eea563be1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:28.724112 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814047b2-223f-40e1-b018-0c5eea563be1 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:28.724915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814047b2-223f-40e1-b018-0c5eea563be1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:28.725360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814047b2-223f-40e1-b018-0c5eea563be1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:28.727390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814047b2-223f-40e1-b018-0c5eea563be1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:28.733484 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 473/906] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:28.741637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85c5f0-6a7c-422c-beee-21f55d7aa719 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:28.913691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85c5f0-6a7c-422c-beee-21f55d7aa719 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:28.951100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a24af5-7855-4373-af68-632a780b7d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:28.951718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a24af5-7855-4373-af68-632a780b7d30 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:28.951998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a24af5-7855-4373-af68-632a780b7d30 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:28.952300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a24af5-7855-4373-af68-632a780b7d30 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:28.952943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a24af5-7855-4373-af68-632a780b7d30 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:28.953248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a24af5-7855-4373-af68-632a780b7d30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:28.954643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a24af5-7855-4373-af68-632a780b7d30 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:28.960542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 434/907] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:28.970752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814047b2-223f-40e1-b018-0c5eea563be1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:29.002786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa638503-68e1-4280-aa58-626f9c2775fc None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:29.003176 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa638503-68e1-4280-aa58-626f9c2775fc None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:29.003356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa638503-68e1-4280-aa58-626f9c2775fc None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:29.005050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 435/908] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:29.011083 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5b90b-18fd-4213-a77b-65e21686f13e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:29.011444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5b90b-18fd-4213-a77b-65e21686f13e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:29.011684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5b90b-18fd-4213-a77b-65e21686f13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:29.031500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-65d5b90b-18fd-4213-a77b-65e21686f13e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:29.032819 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-65d5b90b-18fd-4213-a77b-65e21686f13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:29.060511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:29.128451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 436/909] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:29.171958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814047b2-223f-40e1-b018-0c5eea563be1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:29.208012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:29.242997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca244d7f-ac93-4338-8c15-20b42f47054e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:29.244575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca244d7f-ac93-4338-8c15-20b42f47054e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:29.245346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca244d7f-ac93-4338-8c15-20b42f47054e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:29.246131 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca244d7f-ac93-4338-8c15-20b42f47054e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:29.247712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca244d7f-ac93-4338-8c15-20b42f47054e None admin] RBAC: Adding query filter params (interface=public, service_id=cd2b7ee2b287423184fac4006d13a30b, region_id=RegionOne) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:29.248663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca244d7f-ac93-4338-8c15-20b42f47054e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:29.251603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca244d7f-ac93-4338-8c15-20b42f47054e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:29.266357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 474/910] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 11:41:28 2022] GET /identity/v3/endpoints?service_id=cd2b7ee2b287423184fac4006d13a30b&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:29.272647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5b90b-18fd-4213-a77b-65e21686f13e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:29.409082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca244d7f-ac93-4338-8c15-20b42f47054e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:29.477446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5b90b-18fd-4213-a77b-65e21686f13e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:29.526615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd5650-c875-43a8-b13d-322d1eaa3c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:29.527387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd5650-c875-43a8-b13d-322d1eaa3c1f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:29.527794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd5650-c875-43a8-b13d-322d1eaa3c1f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:29.528346 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd5650-c875-43a8-b13d-322d1eaa3c1f None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:29.529143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd5650-c875-43a8-b13d-322d1eaa3c1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:29.530718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd5650-c875-43a8-b13d-322d1eaa3c1f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:29.544213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 437/911] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 11:41:29 2022] GET /identity/v3/services => generated 1268 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 11:41:29.581167 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca244d7f-ac93-4338-8c15-20b42f47054e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:29.626781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15c233-38fe-4f51-9d49-a79c5b4a8257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:29.627480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c233-38fe-4f51-9d49-a79c5b4a8257 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:29.627773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c233-38fe-4f51-9d49-a79c5b4a8257 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:29.628052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c233-38fe-4f51-9d49-a79c5b4a8257 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:29.629061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15c233-38fe-4f51-9d49-a79c5b4a8257 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:29.630588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15c233-38fe-4f51-9d49-a79c5b4a8257 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:29.631254 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35471c14-0353-4773-ab0e-48ead4af22fa None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:29.631713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35471c14-0353-4773-ab0e-48ead4af22fa None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:29.632161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35471c14-0353-4773-ab0e-48ead4af22fa None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:29.634946 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 438/912] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:29.643300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8df1db-0aca-4392-a693-2236af4c6709 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:29.643745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8df1db-0aca-4392-a693-2236af4c6709 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:29.644226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8df1db-0aca-4392-a693-2236af4c6709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:29.674684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-ac8df1db-0aca-4392-a693-2236af4c6709 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:29.676689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ac8df1db-0aca-4392-a693-2236af4c6709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:29.710986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:29.736308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 475/913] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:41:29 2022] POST /identity/v3/services => generated 241 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:41:29.736308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217d3a2-cb31-4129-8eeb-0631df056258 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:29.736308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217d3a2-cb31-4129-8eeb-0631df056258 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:29.736308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217d3a2-cb31-4129-8eeb-0631df056258 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:29.736308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 476/914] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:29.742608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae9718-63ce-4e08-bcff-fe7704b7e573 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:29.743062 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae9718-63ce-4e08-bcff-fe7704b7e573 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:29.743464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae9718-63ce-4e08-bcff-fe7704b7e573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:29.775094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d2ae9718-63ce-4e08-bcff-fe7704b7e573 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:29.777387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d2ae9718-63ce-4e08-bcff-fe7704b7e573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:29.842227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 439/915] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:29.859247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:29.923018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 477/916] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:30.165948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8df1db-0aca-4392-a693-2236af4c6709 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:30.283238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8df1db-0aca-4392-a693-2236af4c6709 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:30.323004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038bdb6-b00c-4a9c-9531-0c83b9a71054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:30.323657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038bdb6-b00c-4a9c-9531-0c83b9a71054 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:30.323933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038bdb6-b00c-4a9c-9531-0c83b9a71054 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:30.324220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038bdb6-b00c-4a9c-9531-0c83b9a71054 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:30.336941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038bdb6-b00c-4a9c-9531-0c83b9a71054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-e038bdb6-b00c-4a9c-9531-0c83b9a71054 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:30.342353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:30.343479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:30.344444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 440/917] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 11:41:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:30.350526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae9718-63ce-4e08-bcff-fe7704b7e573 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:30.420717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038bdb6-b00c-4a9c-9531-0c83b9a71054 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:30.436610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:30.498078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae9718-63ce-4e08-bcff-fe7704b7e573 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:30.535618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd68260-cca3-4ff0-a424-70456a7da5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:30.536335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68260-cca3-4ff0-a424-70456a7da5f1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:30.536617 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68260-cca3-4ff0-a424-70456a7da5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:30.536889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68260-cca3-4ff0-a424-70456a7da5f1 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:30.537565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd68260-cca3-4ff0-a424-70456a7da5f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:30.537866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd68260-cca3-4ff0-a424-70456a7da5f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:30.539243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd68260-cca3-4ff0-a424-70456a7da5f1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:30.549917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 478/918] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 11:41:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:30.560147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd68260-cca3-4ff0-a424-70456a7da5f1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:30.602444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038bdb6-b00c-4a9c-9531-0c83b9a71054 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:30.608340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:30.646405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ae45d-b807-410b-84ee-4571dd1c084d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:30.646894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ae45d-b807-410b-84ee-4571dd1c084d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:30.647172 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ae45d-b807-410b-84ee-4571dd1c084d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:30.647439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ae45d-b807-410b-84ee-4571dd1c084d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:30.648028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ae45d-b807-410b-84ee-4571dd1c084d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:30.649983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ae45d-b807-410b-84ee-4571dd1c084d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-407ae45d-b807-410b-84ee-4571dd1c084d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:30.659892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:30.661377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 11:41:30.662235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:30.666182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 441/919] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:41:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:30.667185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ae45d-b807-410b-84ee-4571dd1c084d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:30.710354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd68260-cca3-4ff0-a424-70456a7da5f1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:30.747778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271916f-ce55-40d2-8751-ed2818ef4a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:30.749045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271916f-ce55-40d2-8751-ed2818ef4a5e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:30.749708 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271916f-ce55-40d2-8751-ed2818ef4a5e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:30.750192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271916f-ce55-40d2-8751-ed2818ef4a5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:30.762317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271916f-ce55-40d2-8751-ed2818ef4a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:30.763992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271916f-ce55-40d2-8751-ed2818ef4a5e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:30.774177 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-e271916f-ce55-40d2-8751-ed2818ef4a5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:30.780099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:30.788378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:30.789436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:30.789436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:30.789436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:30.789436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 479/920] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:30.791830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271916f-ce55-40d2-8751-ed2818ef4a5e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:30.835569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ae45d-b807-410b-84ee-4571dd1c084d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:30.885156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4615a27-3339-4040-b613-9597c70cf70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:30.886176 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4615a27-3339-4040-b613-9597c70cf70f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:30.886857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4615a27-3339-4040-b613-9597c70cf70f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:30.887407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4615a27-3339-4040-b613-9597c70cf70f None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:30.888337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4615a27-3339-4040-b613-9597c70cf70f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:30.888875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4615a27-3339-4040-b613-9597c70cf70f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:30.890681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4615a27-3339-4040-b613-9597c70cf70f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:30.901582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:30.902250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 442/921] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:41:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:30.903969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4615a27-3339-4040-b613-9597c70cf70f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:30.915633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271916f-ce55-40d2-8751-ed2818ef4a5e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:30.952952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28ed6-987c-456d-84d6-beefb9f2f23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:30.953572 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28ed6-987c-456d-84d6-beefb9f2f23c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:30.953848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28ed6-987c-456d-84d6-beefb9f2f23c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:30.954116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28ed6-987c-456d-84d6-beefb9f2f23c None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:30.954757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e28ed6-987c-456d-84d6-beefb9f2f23c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:30.955057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e28ed6-987c-456d-84d6-beefb9f2f23c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:30.956460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e28ed6-987c-456d-84d6-beefb9f2f23c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:30.974566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 480/922] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:30.975455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28ed6-987c-456d-84d6-beefb9f2f23c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:31.023397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4615a27-3339-4040-b613-9597c70cf70f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:31.060143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba89ec-583b-4c40-a8e6-d02b1a1c4138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:31.061158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba89ec-583b-4c40-a8e6-d02b1a1c4138 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:31.061667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba89ec-583b-4c40-a8e6-d02b1a1c4138 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:31.062198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba89ec-583b-4c40-a8e6-d02b1a1c4138 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:31.063094 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba89ec-583b-4c40-a8e6-d02b1a1c4138 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:31.063612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba89ec-583b-4c40-a8e6-d02b1a1c4138 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:31.065288 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba89ec-583b-4c40-a8e6-d02b1a1c4138 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:31.076154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 443/923] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Oct 4 11:41:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:31.077346 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba89ec-583b-4c40-a8e6-d02b1a1c4138 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:31.077527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28ed6-987c-456d-84d6-beefb9f2f23c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:31.080481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:31.113135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd164ea-7700-4977-89be-c1820f0da3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:31.113771 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd164ea-7700-4977-89be-c1820f0da3db None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:31.114047 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd164ea-7700-4977-89be-c1820f0da3db None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:31.114322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd164ea-7700-4977-89be-c1820f0da3db None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:31.122007 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd164ea-7700-4977-89be-c1820f0da3db None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:31.123463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd164ea-7700-4977-89be-c1820f0da3db None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-ebd164ea-7700-4977-89be-c1820f0da3db None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:31.137274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:31.138315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:31.139284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:31.139284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:31.139284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:31.139284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:31.139284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:31.139284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:31.139284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:31.139284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:31.139284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:31.139284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:31.143333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 481/924] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 11:41:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:31.144510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd164ea-7700-4977-89be-c1820f0da3db None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:31.167160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:31.200443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba89ec-583b-4c40-a8e6-d02b1a1c4138 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:31.240335 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6fef3-4e76-4c6d-8c9b-24db63e40a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:31.241367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6fef3-4e76-4c6d-8c9b-24db63e40a19 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:31.241865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6fef3-4e76-4c6d-8c9b-24db63e40a19 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:31.242351 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6fef3-4e76-4c6d-8c9b-24db63e40a19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:31.243271 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6fef3-4e76-4c6d-8c9b-24db63e40a19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:31.245145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6fef3-4e76-4c6d-8c9b-24db63e40a19 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:31.254777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd164ea-7700-4977-89be-c1820f0da3db None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:31.291711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d407a1-5de0-4c2f-bfb0-cc84607f07d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:31.292389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d407a1-5de0-4c2f-bfb0-cc84607f07d9 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:31.292707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d407a1-5de0-4c2f-bfb0-cc84607f07d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:31.292982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d407a1-5de0-4c2f-bfb0-cc84607f07d9 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:31.293711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d407a1-5de0-4c2f-bfb0-cc84607f07d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:31.294009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d407a1-5de0-4c2f-bfb0-cc84607f07d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:31.295577 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d407a1-5de0-4c2f-bfb0-cc84607f07d9 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:31.326600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 482/925] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 11:41:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:31.327739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d407a1-5de0-4c2f-bfb0-cc84607f07d9 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:31.328480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 444/926] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 11:41:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:31.444429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:31.522424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d407a1-5de0-4c2f-bfb0-cc84607f07d9 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:31.577277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501fb1e-c4cd-41e3-a2b9-72828978a3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:31.578433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501fb1e-c4cd-41e3-a2b9-72828978a3fd None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:31.578715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501fb1e-c4cd-41e3-a2b9-72828978a3fd None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:31.578989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501fb1e-c4cd-41e3-a2b9-72828978a3fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:31.592018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501fb1e-c4cd-41e3-a2b9-72828978a3fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:31.594107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501fb1e-c4cd-41e3-a2b9-72828978a3fd None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-1501fb1e-c4cd-41e3-a2b9-72828978a3fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:31.611254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:31.612511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:31.613445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:31.613445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:31.613445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:31.615542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 483/927] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:31.621459 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6fef3-4e76-4c6d-8c9b-24db63e40a19 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:31.692353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:31.746526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6fef3-4e76-4c6d-8c9b-24db63e40a19 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:31.809994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025f69e-57a2-4dac-8f88-6525831df1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:31.810608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025f69e-57a2-4dac-8f88-6525831df1f5 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:31.810890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025f69e-57a2-4dac-8f88-6525831df1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:31.811161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025f69e-57a2-4dac-8f88-6525831df1f5 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:31.811812 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025f69e-57a2-4dac-8f88-6525831df1f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:31.812113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025f69e-57a2-4dac-8f88-6525831df1f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:31.813519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025f69e-57a2-4dac-8f88-6525831df1f5 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:31.827912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 445/928] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:31.833801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501fb1e-c4cd-41e3-a2b9-72828978a3fd None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:31.988890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501fb1e-c4cd-41e3-a2b9-72828978a3fd None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:32.006369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:32.036620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe6b78-167a-4ce6-9f49-b6e8e5dc4647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:32.037645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe6b78-167a-4ce6-9f49-b6e8e5dc4647 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:32.038102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe6b78-167a-4ce6-9f49-b6e8e5dc4647 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:32.038544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe6b78-167a-4ce6-9f49-b6e8e5dc4647 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:32.048456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe6b78-167a-4ce6-9f49-b6e8e5dc4647 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:32.050230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe6b78-167a-4ce6-9f49-b6e8e5dc4647 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-49fe6b78-167a-4ce6-9f49-b6e8e5dc4647 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:32.063143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:32.064361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:32.065487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:32.065487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:32.065487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:32.065487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:32.065487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:32.065487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:32.065487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:32.065487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:32.067697 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 484/929] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:32.073490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025f69e-57a2-4dac-8f88-6525831df1f5 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:32.141648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43454088-c762-4b7a-87b1-9490ce1c7864 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:32.141912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43454088-c762-4b7a-87b1-9490ce1c7864 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:32.142192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43454088-c762-4b7a-87b1-9490ce1c7864 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:32.143840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 485/930] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:32.150123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb316-8e0c-4359-8e31-4582ff9d3eca None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:32.157520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb316-8e0c-4359-8e31-4582ff9d3eca None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:32.157520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb316-8e0c-4359-8e31-4582ff9d3eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:32.171544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b11eb316-8e0c-4359-8e31-4582ff9d3eca None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:32.172967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b11eb316-8e0c-4359-8e31-4582ff9d3eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:32.256119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 486/931] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:32.261873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce9d0-4858-4919-b5f0-a73d3ebae305 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:32.262165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce9d0-4858-4919-b5f0-a73d3ebae305 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:32.262581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce9d0-4858-4919-b5f0-a73d3ebae305 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:32.264413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 487/932] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:32.270725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025f69e-57a2-4dac-8f88-6525831df1f5 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:32.274998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82418d1f-96f2-4101-9aab-ad43c618c201 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:32.275761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82418d1f-96f2-4101-9aab-ad43c618c201 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:32.276431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82418d1f-96f2-4101-9aab-ad43c618c201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:32.298742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:32.306441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-82418d1f-96f2-4101-9aab-ad43c618c201 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:32.307568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-82418d1f-96f2-4101-9aab-ad43c618c201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:32.324913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7f106-b80a-416d-bc58-6a772bc2804c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:32.326242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7f106-b80a-416d-bc58-6a772bc2804c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:32.326755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7f106-b80a-416d-bc58-6a772bc2804c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:32.327242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7f106-b80a-416d-bc58-6a772bc2804c None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:32.329570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7f106-b80a-416d-bc58-6a772bc2804c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:32.331181 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7f106-b80a-416d-bc58-6a772bc2804c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:32.333801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7f106-b80a-416d-bc58-6a772bc2804c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:32.355746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 446/933] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:32.362682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7f106-b80a-416d-bc58-6a772bc2804c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:32.416221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 488/934] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:32.512537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7f106-b80a-416d-bc58-6a772bc2804c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:32.539329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82418d1f-96f2-4101-9aab-ad43c618c201 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:32.571734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41252bd3-130a-483f-a547-82dc5fca54c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:32.572914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:32.574359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41252bd3-130a-483f-a547-82dc5fca54c6 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:32.575036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41252bd3-130a-483f-a547-82dc5fca54c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:32.575524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41252bd3-130a-483f-a547-82dc5fca54c6 None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/952ec01f4a174b6d94c3afd98f90ba81/roles/87a27c6c32d94ae7b373df89e15cb25e` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:32.578745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41252bd3-130a-483f-a547-82dc5fca54c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=952ec01f4a174b6d94c3afd98f90ba81, role_id=87a27c6c32d94ae7b373df89e15cb25e)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:32.581029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41252bd3-130a-483f-a547-82dc5fca54c6 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:32.666123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82418d1f-96f2-4101-9aab-ad43c618c201 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:32.686799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 447/935] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:41:32 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/952ec01f4a174b6d94c3afd98f90ba81/roles/87a27c6c32d94ae7b373df89e15cb25e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 11:41:32.729485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c926248-a6df-4641-9f04-93810a9d417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:32.730150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c926248-a6df-4641-9f04-93810a9d417f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:32.730574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c926248-a6df-4641-9f04-93810a9d417f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:32.730841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c926248-a6df-4641-9f04-93810a9d417f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:32.731538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c926248-a6df-4641-9f04-93810a9d417f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:32.733426 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c926248-a6df-4641-9f04-93810a9d417f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-0c926248-a6df-4641-9f04-93810a9d417f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:32.748359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:32.750101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:32.751519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 489/936] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:32.762867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41252bd3-130a-483f-a547-82dc5fca54c6 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:32.831470 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c926248-a6df-4641-9f04-93810a9d417f None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:32.850617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:32.956742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41252bd3-130a-483f-a547-82dc5fca54c6 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:33.004799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:33.014611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c926248-a6df-4641-9f04-93810a9d417f None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:33.023385 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934101df-8039-4181-bbcf-474204ef3a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:33.025264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934101df-8039-4181-bbcf-474204ef3a33 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:33.026033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934101df-8039-4181-bbcf-474204ef3a33 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:33.027495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934101df-8039-4181-bbcf-474204ef3a33 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:33.029189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934101df-8039-4181-bbcf-474204ef3a33 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:33.029921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934101df-8039-4181-bbcf-474204ef3a33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:33.034807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934101df-8039-4181-bbcf-474204ef3a33 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:33.047182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 448/937] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 11:41:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:33.055535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934101df-8039-4181-bbcf-474204ef3a33 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:33.073777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1da69d-f30a-41ec-9fc2-5e94dc926f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:33.075136 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1da69d-f30a-41ec-9fc2-5e94dc926f39 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:33.075600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1da69d-f30a-41ec-9fc2-5e94dc926f39 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:33.076246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1da69d-f30a-41ec-9fc2-5e94dc926f39 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:33.077271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1da69d-f30a-41ec-9fc2-5e94dc926f39 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:33.079001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1da69d-f30a-41ec-9fc2-5e94dc926f39 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-bf1da69d-f30a-41ec-9fc2-5e94dc926f39 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:33.095881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:33.097663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 11:41:33.099391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:33.104281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 490/938] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Oct 4 11:41:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:33.111644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1da69d-f30a-41ec-9fc2-5e94dc926f39 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:33.271504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934101df-8039-4181-bbcf-474204ef3a33 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:33.276797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:33.303661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1da69d-f30a-41ec-9fc2-5e94dc926f39 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:33.350602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ee6c3-1c2c-4d6a-a591-7d447149a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:33.351110 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ee6c3-1c2c-4d6a-a591-7d447149a3df None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:33.351451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ee6c3-1c2c-4d6a-a591-7d447149a3df None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:33.352422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ee6c3-1c2c-4d6a-a591-7d447149a3df None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:33.353116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ee6c3-1c2c-4d6a-a591-7d447149a3df None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:33.353421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ee6c3-1c2c-4d6a-a591-7d447149a3df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:33.354786 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ee6c3-1c2c-4d6a-a591-7d447149a3df None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:33.355506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e40a7-3973-4456-9d01-2725be2f8ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:33.357894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e40a7-3973-4456-9d01-2725be2f8ca7 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:33.358626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e40a7-3973-4456-9d01-2725be2f8ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:33.359544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e40a7-3973-4456-9d01-2725be2f8ca7 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:33.360993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e40a7-3973-4456-9d01-2725be2f8ca7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:33.366969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e40a7-3973-4456-9d01-2725be2f8ca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:33.367166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 491/939] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:33.367328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e40a7-3973-4456-9d01-2725be2f8ca7 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:33.371755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ee6c3-1c2c-4d6a-a591-7d447149a3df None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:33.374905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 449/940] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 11:41:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:33.380703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e40a7-3973-4456-9d01-2725be2f8ca7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:33.425816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:33.510959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e40a7-3973-4456-9d01-2725be2f8ca7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:33.554312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16acc6f-d14f-4492-bbe8-da77861bfc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:33.555541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16acc6f-d14f-4492-bbe8-da77861bfc55 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:33.555836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16acc6f-d14f-4492-bbe8-da77861bfc55 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:33.556130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16acc6f-d14f-4492-bbe8-da77861bfc55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:33.557280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16acc6f-d14f-4492-bbe8-da77861bfc55 None admin] RBAC: Adding query filter params (interface=public, service_id=30e0a83e72564e81a449524728f04ae5, region_id=RegionOne) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:33.557431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16acc6f-d14f-4492-bbe8-da77861bfc55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:33.558884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16acc6f-d14f-4492-bbe8-da77861bfc55 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:33.565775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 450/941] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 11:41:33 2022] GET /identity/v3/endpoints?service_id=30e0a83e72564e81a449524728f04ae5&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:33.572475 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16acc6f-d14f-4492-bbe8-da77861bfc55 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:33.588341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ee6c3-1c2c-4d6a-a591-7d447149a3df None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:33.634186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661030da-6910-4c20-a806-9d73ed15910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:33.634733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661030da-6910-4c20-a806-9d73ed15910c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:33.635097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661030da-6910-4c20-a806-9d73ed15910c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:33.635258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661030da-6910-4c20-a806-9d73ed15910c None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:33.635969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661030da-6910-4c20-a806-9d73ed15910c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:33.636310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661030da-6910-4c20-a806-9d73ed15910c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:33.637729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661030da-6910-4c20-a806-9d73ed15910c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:33.638633 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:33.644154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 492/942] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:33.652538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661030da-6910-4c20-a806-9d73ed15910c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:33.780707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16acc6f-d14f-4492-bbe8-da77861bfc55 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:33.805811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:33.826118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661030da-6910-4c20-a806-9d73ed15910c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:33.829242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19478e13-6fb3-4f77-a587-2d6e113c608d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:33.829917 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19478e13-6fb3-4f77-a587-2d6e113c608d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:33.831360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19478e13-6fb3-4f77-a587-2d6e113c608d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:33.831360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19478e13-6fb3-4f77-a587-2d6e113c608d None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:33.831360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19478e13-6fb3-4f77-a587-2d6e113c608d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:33.832470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19478e13-6fb3-4f77-a587-2d6e113c608d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:33.842879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 451/943] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 11:41:33 2022] GET /identity/v3/services => generated 1497 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 11:41:33.845268 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74334ad6-c792-47c4-8b8a-7d2b0afbb851 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:33.845704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74334ad6-c792-47c4-8b8a-7d2b0afbb851 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:33.846216 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74334ad6-c792-47c4-8b8a-7d2b0afbb851 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:33.852693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 452/944] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:33.858352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5b7c8-4858-4db0-acec-eecf8101f427 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:33.858853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5b7c8-4858-4db0-acec-eecf8101f427 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:33.859289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5b7c8-4858-4db0-acec-eecf8101f427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:33.890353 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8857063-844f-48aa-8799-bff8ce798777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:33.890782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8857063-844f-48aa-8799-bff8ce798777 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:33.891049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8857063-844f-48aa-8799-bff8ce798777 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:33.891327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8857063-844f-48aa-8799-bff8ce798777 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:33.892033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8857063-844f-48aa-8799-bff8ce798777 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:33.894065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8857063-844f-48aa-8799-bff8ce798777 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:33.899611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f8d5b7c8-4858-4db0-acec-eecf8101f427 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:33.901623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f8d5b7c8-4858-4db0-acec-eecf8101f427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:34.008634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 493/945] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 11:41:33 2022] POST /identity/v3/endpoints => generated 347 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:34.024460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 453/946] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:34.303934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8857063-844f-48aa-8799-bff8ce798777 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:34.479503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8857063-844f-48aa-8799-bff8ce798777 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:34.546440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:34.547419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:34.547891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:34.548420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:34.549643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:34.552681 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:34.558042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: WARNING keystone.server.flask.common [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 11:41:34.585217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:34.625897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.sql.core [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 11:41:34.625897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 11:41:34.625897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [parameters: {'id': '6dbce3de9d9d4041a6828a2e172947a2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 11:41:34.625897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93212) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6dbce3de9d9d4041a6828a2e172947a2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6dbce3de9d9d4041a6828a2e172947a2. Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application conn.query(q) Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 11:41:34.640092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application result.read() Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 04 11:41:34.641751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application next(self.gen) Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application yield resource Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application next(self.gen) Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise self.value Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application session.commit() Oct 04 11:41:34.643643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.session.flush() Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application compat.raise_( Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 11:41:34.647296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 11:41:34.655090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application conn.query(q) Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application result.read() Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 11:41:34.657694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [parameters: {'id': '6dbce3de9d9d4041a6828a2e172947a2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:34.660416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:34.665864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:34.669954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:34.669954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 04 11:41:34.669954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 04 11:41:34.669954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:34.669954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:34.669954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 04 11:41:34.669954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 04 11:41:34.669954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6dbce3de9d9d4041a6828a2e172947a2. Oct 04 11:41:34.669954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:34.669954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 494/947] 10.222.0.15 () {60 vars in 1140 bytes} [Tue Oct 4 11:41:34 2022] POST /identity/v3/projects => generated 262 bytes in 340 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:34.679394 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5b7c8-4858-4db0-acec-eecf8101f427 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:34.718491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:34.835714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5b7c8-4858-4db0-acec-eecf8101f427 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:34.879977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9fcd3-e8c9-4b1e-bf0e-25f955f49459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:34.880677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9fcd3-e8c9-4b1e-bf0e-25f955f49459 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:34.880955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9fcd3-e8c9-4b1e-bf0e-25f955f49459 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:34.881240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9fcd3-e8c9-4b1e-bf0e-25f955f49459 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:34.889705 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9fcd3-e8c9-4b1e-bf0e-25f955f49459 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:34.891259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9fcd3-e8c9-4b1e-bf0e-25f955f49459 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-f7c9fcd3-e8c9-4b1e-bf0e-25f955f49459 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:34.904274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:34.906005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:34.907671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:34.907671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:34.907671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:34.907671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:34.907671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:34.907671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:34.907671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:34.907671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:34.907671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 454/948] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:34.920373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:35.082420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e01633-48e8-4934-8059-7c4611780358 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:35.082839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e01633-48e8-4934-8059-7c4611780358 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:35.082996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e01633-48e8-4934-8059-7c4611780358 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:35.084740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 455/949] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:35.091137 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d5cc-156e-489c-8fde-24d84cff0cf0 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:35.091363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d5cc-156e-489c-8fde-24d84cff0cf0 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:35.091664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d5cc-156e-489c-8fde-24d84cff0cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:35.112025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d028d5cc-156e-489c-8fde-24d84cff0cf0 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:35.113422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d028d5cc-156e-489c-8fde-24d84cff0cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:35.135188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f71083-ecf8-4d21-a31c-9c9adaddd2a4 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:35.200675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec6213-23d9-4b7f-ae8e-ecb21f1069e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:35.201304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec6213-23d9-4b7f-ae8e-ecb21f1069e6 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:35.201588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec6213-23d9-4b7f-ae8e-ecb21f1069e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:35.201877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec6213-23d9-4b7f-ae8e-ecb21f1069e6 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:35.202617 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec6213-23d9-4b7f-ae8e-ecb21f1069e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:35.202921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec6213-23d9-4b7f-ae8e-ecb21f1069e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:35.204514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 456/950] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:35.204822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec6213-23d9-4b7f-ae8e-ecb21f1069e6 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:35.230463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 495/951] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:34 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:35.705603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028d5cc-156e-489c-8fde-24d84cff0cf0 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:35.727878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:35.897778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028d5cc-156e-489c-8fde-24d84cff0cf0 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:35.964777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38ef15-3f11-4115-81e2-0234f97d7f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:35.965753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38ef15-3f11-4115-81e2-0234f97d7f1c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:35.966245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38ef15-3f11-4115-81e2-0234f97d7f1c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:35.966741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38ef15-3f11-4115-81e2-0234f97d7f1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:35.988105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38ef15-3f11-4115-81e2-0234f97d7f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-bd38ef15-3f11-4115-81e2-0234f97d7f1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:35.998880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:35.999966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:41:36.000968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:36.001654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 457/952] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 11:41:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:36.012349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec6213-23d9-4b7f-ae8e-ecb21f1069e6 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:36.129394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:36.205670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec6213-23d9-4b7f-ae8e-ecb21f1069e6 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:36.246563 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb3e75-b7c9-4b46-9631-9edb697c1b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:36.247120 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb3e75-b7c9-4b46-9631-9edb697c1b3c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:36.247405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb3e75-b7c9-4b46-9631-9edb697c1b3c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:36.247682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb3e75-b7c9-4b46-9631-9edb697c1b3c None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:36.248416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb3e75-b7c9-4b46-9631-9edb697c1b3c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:36.248726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb3e75-b7c9-4b46-9631-9edb697c1b3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:36.250247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb3e75-b7c9-4b46-9631-9edb697c1b3c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:36.260445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 496/953] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 11:41:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:36.269280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38ef15-3f11-4115-81e2-0234f97d7f1c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:36.372609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:36.405635 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38ef15-3f11-4115-81e2-0234f97d7f1c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:36.421706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f1c3-83bd-4eee-a843-da44678253d7 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:36.422033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f1c3-83bd-4eee-a843-da44678253d7 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:36.422586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f1c3-83bd-4eee-a843-da44678253d7 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:36.424271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 497/954] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:36.433654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca45dec-2555-4ade-b0da-fa6069fb1bd4 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:36.433992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca45dec-2555-4ade-b0da-fa6069fb1bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:36.434279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca45dec-2555-4ade-b0da-fa6069fb1bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:36.450661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcb074-df6f-42fa-8a56-5e873748e9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:36.451709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcb074-df6f-42fa-8a56-5e873748e9a8 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:36.452243 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcb074-df6f-42fa-8a56-5e873748e9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:36.452746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcb074-df6f-42fa-8a56-5e873748e9a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:36.459444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-cca45dec-2555-4ade-b0da-fa6069fb1bd4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:36.461270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cca45dec-2555-4ade-b0da-fa6069fb1bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:36.465349 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcb074-df6f-42fa-8a56-5e873748e9a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:36.467105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcb074-df6f-42fa-8a56-5e873748e9a8 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-eafcb074-df6f-42fa-8a56-5e873748e9a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:36.485071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:36.486127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:36.487060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:36.487060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:36.487060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:36.489429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 458/955] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:36.502298 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcb074-df6f-42fa-8a56-5e873748e9a8 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:36.552815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 498/956] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:36.629841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcb074-df6f-42fa-8a56-5e873748e9a8 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:36.655432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:36.667779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79338da1-5705-4acc-ad51-79ab3d022366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:36.668421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79338da1-5705-4acc-ad51-79ab3d022366 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:36.668692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79338da1-5705-4acc-ad51-79ab3d022366 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:36.668961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79338da1-5705-4acc-ad51-79ab3d022366 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:36.669600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79338da1-5705-4acc-ad51-79ab3d022366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:36.669899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79338da1-5705-4acc-ad51-79ab3d022366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:36.671301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79338da1-5705-4acc-ad51-79ab3d022366 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:36.692097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca45dec-2555-4ade-b0da-fa6069fb1bd4 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:36.693008 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 459/957] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:36.763912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:36.812967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f8cc-9874-4c69-bf7f-c7c24c7bcff4 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:36.813756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f8cc-9874-4c69-bf7f-c7c24c7bcff4 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:36.813756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f8cc-9874-4c69-bf7f-c7c24c7bcff4 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:36.815128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 460/958] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:36.820568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27f9ad-e9e7-4162-97ef-55a46543b301 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:36.820853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27f9ad-e9e7-4162-97ef-55a46543b301 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:36.821135 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27f9ad-e9e7-4162-97ef-55a46543b301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:36.840817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-0e27f9ad-e9e7-4162-97ef-55a46543b301 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:36.842036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0e27f9ad-e9e7-4162-97ef-55a46543b301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:36.859355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca45dec-2555-4ade-b0da-fa6069fb1bd4 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:36.902341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd38149-a8eb-4b66-b075-f4389d8cca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:36.903254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd38149-a8eb-4b66-b075-f4389d8cca97 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:36.903254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd38149-a8eb-4b66-b075-f4389d8cca97 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:36.903584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd38149-a8eb-4b66-b075-f4389d8cca97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:36.912087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd38149-a8eb-4b66-b075-f4389d8cca97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:36.913674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd38149-a8eb-4b66-b075-f4389d8cca97 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-3fd38149-a8eb-4b66-b075-f4389d8cca97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:36.929519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:36.931906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:36.933836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 499/959] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 11:41:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:36.938358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd38149-a8eb-4b66-b075-f4389d8cca97 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:36.996590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 461/960] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:37.045926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27f9ad-e9e7-4162-97ef-55a46543b301 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:37.137737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd38149-a8eb-4b66-b075-f4389d8cca97 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:37.158043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:37.199464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37378c25-24e1-4728-b603-14eab8b39bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:37.200465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37378c25-24e1-4728-b603-14eab8b39bf6 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:37.200939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37378c25-24e1-4728-b603-14eab8b39bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:37.201409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37378c25-24e1-4728-b603-14eab8b39bf6 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:37.202706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37378c25-24e1-4728-b603-14eab8b39bf6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:37.203224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37378c25-24e1-4728-b603-14eab8b39bf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:37.205781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37378c25-24e1-4728-b603-14eab8b39bf6 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:37.241878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27f9ad-e9e7-4162-97ef-55a46543b301 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:37.244745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 500/961] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Oct 4 11:41:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:37.250286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37378c25-24e1-4728-b603-14eab8b39bf6 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:37.269069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:37.302640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648c927-97b5-4aa7-b835-b0214a3a5322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:37.305428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648c927-97b5-4aa7-b835-b0214a3a5322 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:37.306272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648c927-97b5-4aa7-b835-b0214a3a5322 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:37.307093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648c927-97b5-4aa7-b835-b0214a3a5322 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:37.308398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648c927-97b5-4aa7-b835-b0214a3a5322 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:37.310901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648c927-97b5-4aa7-b835-b0214a3a5322 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-1648c927-97b5-4aa7-b835-b0214a3a5322 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:37.326086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:37.327261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:37.328637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 462/962] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:37.329503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648c927-97b5-4aa7-b835-b0214a3a5322 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:37.412291 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37378c25-24e1-4728-b603-14eab8b39bf6 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:37.456265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3088e5-a8dc-43ef-826c-f65eee7c385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:37.457222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3088e5-a8dc-43ef-826c-f65eee7c385d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:37.457670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3088e5-a8dc-43ef-826c-f65eee7c385d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:37.458037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3088e5-a8dc-43ef-826c-f65eee7c385d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:37.482744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648c927-97b5-4aa7-b835-b0214a3a5322 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:37.516603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:37.537648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f26ad-ec0b-4b56-85f1-a7d7ae635d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:37.538629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f26ad-ec0b-4b56-85f1-a7d7ae635d12 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:37.539743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f26ad-ec0b-4b56-85f1-a7d7ae635d12 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:37.540244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f26ad-ec0b-4b56-85f1-a7d7ae635d12 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:37.541219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f26ad-ec0b-4b56-85f1-a7d7ae635d12 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:37.543581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f26ad-ec0b-4b56-85f1-a7d7ae635d12 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-8d0f26ad-ec0b-4b56-85f1-a7d7ae635d12 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:37.558599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:37.559667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:37.560551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 463/963] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:37.562169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f26ad-ec0b-4b56-85f1-a7d7ae635d12 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:37.582274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3088e5-a8dc-43ef-826c-f65eee7c385d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:37.585230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3088e5-a8dc-43ef-826c-f65eee7c385d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:37.627407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:37.706920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f26ad-ec0b-4b56-85f1-a7d7ae635d12 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:37.733778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 501/964] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:37 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 11:41:37.733778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3088e5-a8dc-43ef-826c-f65eee7c385d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:37.756743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43097ad9-cc08-4134-b383-9869b7f3e049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:37.757329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43097ad9-cc08-4134-b383-9869b7f3e049 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:37.757620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43097ad9-cc08-4134-b383-9869b7f3e049 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:37.758394 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43097ad9-cc08-4134-b383-9869b7f3e049 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:37.759021 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43097ad9-cc08-4134-b383-9869b7f3e049 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:37.759319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43097ad9-cc08-4134-b383-9869b7f3e049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:37.760788 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43097ad9-cc08-4134-b383-9869b7f3e049 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:37.767186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 464/965] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:37.768012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43097ad9-cc08-4134-b383-9869b7f3e049 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:37.828343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:37.877680 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3088e5-a8dc-43ef-826c-f65eee7c385d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:37.914374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43097ad9-cc08-4134-b383-9869b7f3e049 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:37.918072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:37.932369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59422e-4148-4757-89f2-9ea0721d99db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:37.933308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59422e-4148-4757-89f2-9ea0721d99db None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:37.933659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59422e-4148-4757-89f2-9ea0721d99db None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:37.934086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59422e-4148-4757-89f2-9ea0721d99db None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:37.934815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59422e-4148-4757-89f2-9ea0721d99db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:37.935175 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59422e-4148-4757-89f2-9ea0721d99db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:37.936964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59422e-4148-4757-89f2-9ea0721d99db None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:37.942714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 502/966] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 11:41:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:37.943766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59422e-4148-4757-89f2-9ea0721d99db None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:37.959676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e6da8-7de4-4a56-9cbe-fd5465c69d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:37.960467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e6da8-7de4-4a56-9cbe-fd5465c69d77 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:37.960964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e6da8-7de4-4a56-9cbe-fd5465c69d77 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:37.962052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e6da8-7de4-4a56-9cbe-fd5465c69d77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:37.980282 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e6da8-7de4-4a56-9cbe-fd5465c69d77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:37.981924 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e6da8-7de4-4a56-9cbe-fd5465c69d77 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-133e6da8-7de4-4a56-9cbe-fd5465c69d77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:37.998890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:38.001169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:38.002996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:38.002996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:38.002996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:38.002996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 465/967] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:38.002996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42389-e892-4aff-b385-74e4fa652676 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:38.002996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42389-e892-4aff-b385-74e4fa652676 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:38.002996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42389-e892-4aff-b385-74e4fa652676 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:38.003802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 466/968] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:38.004736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f42389-e892-4aff-b385-74e4fa652676 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:38.087948 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59422e-4148-4757-89f2-9ea0721d99db None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:38.125989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:38.127264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f42389-e892-4aff-b385-74e4fa652676 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:38.131534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254e874-f552-4be8-ad96-fabfa3a3ff2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:38.132321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254e874-f552-4be8-ad96-fabfa3a3ff2a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:38.132824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254e874-f552-4be8-ad96-fabfa3a3ff2a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:38.133204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254e874-f552-4be8-ad96-fabfa3a3ff2a None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:38.133942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254e874-f552-4be8-ad96-fabfa3a3ff2a None admin] RBAC: Adding query filter params (type=compute) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:38.134341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254e874-f552-4be8-ad96-fabfa3a3ff2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:38.135913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254e874-f552-4be8-ad96-fabfa3a3ff2a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:38.156493 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 503/969] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:38.156493 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254e874-f552-4be8-ad96-fabfa3a3ff2a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:38.172926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332724c4-8182-40bf-8843-bb0dd0eb62f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:38.173795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332724c4-8182-40bf-8843-bb0dd0eb62f2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:38.174245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332724c4-8182-40bf-8843-bb0dd0eb62f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:38.174660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332724c4-8182-40bf-8843-bb0dd0eb62f2 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:38.175443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332724c4-8182-40bf-8843-bb0dd0eb62f2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:38.175875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332724c4-8182-40bf-8843-bb0dd0eb62f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:38.177684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332724c4-8182-40bf-8843-bb0dd0eb62f2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:38.184694 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 467/970] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 11:41:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:38.186310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f99fe-80e0-4cc2-8bb2-b40820292e31 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:38.186723 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f99fe-80e0-4cc2-8bb2-b40820292e31 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:38.187177 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f99fe-80e0-4cc2-8bb2-b40820292e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:38.222245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-c64f99fe-80e0-4cc2-8bb2-b40820292e31 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:38.224112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c64f99fe-80e0-4cc2-8bb2-b40820292e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:38.248757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:38.291049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254e874-f552-4be8-ad96-fabfa3a3ff2a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:38.333765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f14c07-12ef-4956-8905-d6ad1d397f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:38.334714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f14c07-12ef-4956-8905-d6ad1d397f7f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:38.335129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f14c07-12ef-4956-8905-d6ad1d397f7f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:38.335562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f14c07-12ef-4956-8905-d6ad1d397f7f None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:38.337123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f14c07-12ef-4956-8905-d6ad1d397f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:38.337590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f14c07-12ef-4956-8905-d6ad1d397f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:38.339296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f14c07-12ef-4956-8905-d6ad1d397f7f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:38.342491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 468/971] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:38.343914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f99fe-80e0-4cc2-8bb2-b40820292e31 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:38.369419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 504/972] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:38.377944 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f14c07-12ef-4956-8905-d6ad1d397f7f None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:38.509736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:38.559889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f99fe-80e0-4cc2-8bb2-b40820292e31 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:38.565091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f14c07-12ef-4956-8905-d6ad1d397f7f None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:38.620270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60258317-b149-4dbc-bc24-39f9ecfcf0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:38.620915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60258317-b149-4dbc-bc24-39f9ecfcf0a0 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:38.621186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60258317-b149-4dbc-bc24-39f9ecfcf0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:38.621501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60258317-b149-4dbc-bc24-39f9ecfcf0a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:38.622121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60258317-b149-4dbc-bc24-39f9ecfcf0a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:38.623500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60258317-b149-4dbc-bc24-39f9ecfcf0a0 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:38.623676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120c1d1-bf76-4e07-820e-b1cf32f9e9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:38.624783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120c1d1-bf76-4e07-820e-b1cf32f9e9ee None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:38.625344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120c1d1-bf76-4e07-820e-b1cf32f9e9ee None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:38.625862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120c1d1-bf76-4e07-820e-b1cf32f9e9ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:38.640915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120c1d1-bf76-4e07-820e-b1cf32f9e9ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:38.643675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120c1d1-bf76-4e07-820e-b1cf32f9e9ee None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-6120c1d1-bf76-4e07-820e-b1cf32f9e9ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:38.666427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:38.667942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:38.669892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:38.669892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:38.669892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:38.669892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:38.669892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:38.669892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:38.669892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:38.669892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:38.676501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 505/973] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:38.678797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120c1d1-bf76-4e07-820e-b1cf32f9e9ee None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:38.718415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 469/974] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 11:41:38 2022] POST /identity/v3/endpoints => generated 349 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:38.826012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60258317-b149-4dbc-bc24-39f9ecfcf0a0 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:38.843847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:38.901149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120c1d1-bf76-4e07-820e-b1cf32f9e9ee None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:38.940618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:38.993327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd2413-5d33-461e-9f1e-6ccc1f565352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:38.995373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd2413-5d33-461e-9f1e-6ccc1f565352 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:38.996650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd2413-5d33-461e-9f1e-6ccc1f565352 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:38.997486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd2413-5d33-461e-9f1e-6ccc1f565352 None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:38.998991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd2413-5d33-461e-9f1e-6ccc1f565352 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:38.999794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd2413-5d33-461e-9f1e-6ccc1f565352 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:39.002854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd2413-5d33-461e-9f1e-6ccc1f565352 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:39.019609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 506/975] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:39.031033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd2413-5d33-461e-9f1e-6ccc1f565352 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:39.043567 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60258317-b149-4dbc-bc24-39f9ecfcf0a0 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:39.103907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:39.105141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:39.105684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:39.106149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:39.107316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:39.110506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:39.115235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WARNING keystone.server.flask.common [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 11:41:39.163860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd2413-5d33-461e-9f1e-6ccc1f565352 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:39.167913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.sql.core [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 11:41:39.167913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 11:41:39.167913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [parameters: {'user_id': 'ac59b873654242229cfa8c2fc5ba0fc7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 11:41:39.167913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93213) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 11:41:39.205828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e1ce9-8542-49bd-888a-2d646218cbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:39.206852 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e1ce9-8542-49bd-888a-2d646218cbdb None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:39.207347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e1ce9-8542-49bd-888a-2d646218cbdb None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:39.207824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e1ce9-8542-49bd-888a-2d646218cbdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:39.208777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e1ce9-8542-49bd-888a-2d646218cbdb None admin] RBAC: Adding query filter params (role.id=87a27c6c32d94ae7b373df89e15cb25e, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=952ec01f4a174b6d94c3afd98f90ba81) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:39.211313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e1ce9-8542-49bd-888a-2d646218cbdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:39.211313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e1ce9-8542-49bd-888a-2d646218cbdb None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application conn.query(q) Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application result.read() Oct 04 11:41:39.215087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 11:41:39.216031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application next(self.gen) Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application yield resource Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application next(self.gen) Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise self.value Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application session.commit() Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 11:41:39.217194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self.session.flush() Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application compat.raise_( Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 11:41:39.218131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 11:41:39.219089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application conn.query(q) Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application result.read() Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 11:41:39.219994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ac59b873654242229cfa8c2fc5ba0fc7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:39.220903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 04 11:41:39.221865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 04 11:41:39.224211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 04 11:41:39.224211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 04 11:41:39.224211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 11:41:39.224211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 11:41:39.224211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 11:41:39.224211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:39.224211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 470/976] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Oct 4 11:41:38 2022] POST /identity/v3/users => generated 160 bytes in 392 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:39.228814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:39.229960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 507/977] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:41:39 2022] GET /identity/v3/role_assignments?user.id=952ec01f4a174b6d94c3afd98f90ba81&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=87a27c6c32d94ae7b373df89e15cb25e => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:39.339028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:39.382554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4842f-4002-4604-b346-1957bb7c5f51 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:39.426240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012906c-0b92-4b6b-823c-f1f75b0ace98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:39.427308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012906c-0b92-4b6b-823c-f1f75b0ace98 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:39.427813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012906c-0b92-4b6b-823c-f1f75b0ace98 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:39.428416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012906c-0b92-4b6b-823c-f1f75b0ace98 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:39.440556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012906c-0b92-4b6b-823c-f1f75b0ace98 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:39.442912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012906c-0b92-4b6b-823c-f1f75b0ace98 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-c012906c-0b92-4b6b-823c-f1f75b0ace98 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:39.456912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:39.457983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:39.458943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:39.458943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:39.458943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:39.458943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:39.458943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:39.458943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:39.458943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:39.458943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:39.458943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 11:41:39.458943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:39.461858 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 471/978] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:41:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:39.469035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e1ce9-8542-49bd-888a-2d646218cbdb None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:39.518920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:39.639048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e1ce9-8542-49bd-888a-2d646218cbdb None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:39.701861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38053d2d-4386-4c4a-baf8-b0aa34c59ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:39.703332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38053d2d-4386-4c4a-baf8-b0aa34c59ea4 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:39.704119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38053d2d-4386-4c4a-baf8-b0aa34c59ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:39.704917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38053d2d-4386-4c4a-baf8-b0aa34c59ea4 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:39.706473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38053d2d-4386-4c4a-baf8-b0aa34c59ea4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:39.707296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38053d2d-4386-4c4a-baf8-b0aa34c59ea4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:39.710492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38053d2d-4386-4c4a-baf8-b0aa34c59ea4 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:39.781279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 508/979] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Oct 4 11:41:39 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:40.628420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90921-e26b-42f9-b62f-a1eb6d0bc2c4 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:40.628968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90921-e26b-42f9-b62f-a1eb6d0bc2c4 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:40.628968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90921-e26b-42f9-b62f-a1eb6d0bc2c4 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:40.633391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 472/980] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:40.639058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedcb34-e49b-480e-8907-1735a1681bf4 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:40.639795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedcb34-e49b-480e-8907-1735a1681bf4 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:40.639795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedcb34-e49b-480e-8907-1735a1681bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:40.659600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1fedcb34-e49b-480e-8907-1735a1681bf4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:40.660868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1fedcb34-e49b-480e-8907-1735a1681bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:40.740732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 509/981] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:41.007055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d90921-e26b-42f9-b62f-a1eb6d0bc2c4 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:41.122566 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d90921-e26b-42f9-b62f-a1eb6d0bc2c4 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:41.167966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:41.181373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d2494-21fc-44d8-9a34-ae6e7dd92251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:41.182014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d2494-21fc-44d8-9a34-ae6e7dd92251 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:41.182453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d2494-21fc-44d8-9a34-ae6e7dd92251 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:41.183097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d2494-21fc-44d8-9a34-ae6e7dd92251 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:41.184056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d2494-21fc-44d8-9a34-ae6e7dd92251 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:41.186606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d2494-21fc-44d8-9a34-ae6e7dd92251 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:41.262507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 473/982] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:41:41 2022] POST /identity/v3/services => generated 242 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:41.955777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827499-cfd7-49a2-8471-e9493bb713d1 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:41.956395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827499-cfd7-49a2-8471-e9493bb713d1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:41.956796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827499-cfd7-49a2-8471-e9493bb713d1 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:41.959253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 510/983] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:41.965199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25d49-600e-443b-9c44-6fec98f1eabc None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:41.965986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25d49-600e-443b-9c44-6fec98f1eabc None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:41.965986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25d49-600e-443b-9c44-6fec98f1eabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:41.984970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-40d25d49-600e-443b-9c44-6fec98f1eabc None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:41.986530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-40d25d49-600e-443b-9c44-6fec98f1eabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:42.081865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 474/984] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:42.261810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063c48b-e5cd-48ca-807e-b716162d0813 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:42.262101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063c48b-e5cd-48ca-807e-b716162d0813 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:42.262436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063c48b-e5cd-48ca-807e-b716162d0813 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:42.263924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 511/985] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:42.270408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaec13-234b-4311-84e7-2e36f8348a80 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:42.271007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaec13-234b-4311-84e7-2e36f8348a80 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:42.271483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaec13-234b-4311-84e7-2e36f8348a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:42.298442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-2baaec13-234b-4311-84e7-2e36f8348a80 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:42.300227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2baaec13-234b-4311-84e7-2e36f8348a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:42.376810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 475/986] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:42.389565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063c48b-e5cd-48ca-807e-b716162d0813 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:42.426208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:42.538034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063c48b-e5cd-48ca-807e-b716162d0813 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:42.600419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9f80b-e0f7-4131-aafa-b49f1a45a5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:42.601528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9f80b-e0f7-4131-aafa-b49f1a45a5b8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:42.602025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9f80b-e0f7-4131-aafa-b49f1a45a5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:42.602477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9f80b-e0f7-4131-aafa-b49f1a45a5b8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:42.603516 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9f80b-e0f7-4131-aafa-b49f1a45a5b8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:42.606049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9f80b-e0f7-4131-aafa-b49f1a45a5b8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-fdc9f80b-e0f7-4131-aafa-b49f1a45a5b8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:42.624590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:42.625664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:42.628787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 512/987] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:42.633297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baaec13-234b-4311-84e7-2e36f8348a80 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:42.647147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9f80b-e0f7-4131-aafa-b49f1a45a5b8 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:42.697509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:42.777103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baaec13-234b-4311-84e7-2e36f8348a80 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:42.797859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:42.814654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9f80b-e0f7-4131-aafa-b49f1a45a5b8 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:42.840430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694e39d-cd96-4324-8187-326372a963f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:42.841911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694e39d-cd96-4324-8187-326372a963f7 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:42.842194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694e39d-cd96-4324-8187-326372a963f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:42.842477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694e39d-cd96-4324-8187-326372a963f7 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:42.843127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694e39d-cd96-4324-8187-326372a963f7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:42.843436 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694e39d-cd96-4324-8187-326372a963f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:42.844907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694e39d-cd96-4324-8187-326372a963f7 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:42.850663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 476/988] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 11:41:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:42.856637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694e39d-cd96-4324-8187-326372a963f7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:42.864812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f924ba-4bc8-4eae-a26e-31d11597eecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:42.866255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f924ba-4bc8-4eae-a26e-31d11597eecf None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:42.866750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f924ba-4bc8-4eae-a26e-31d11597eecf None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:42.867763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f924ba-4bc8-4eae-a26e-31d11597eecf None admin] PATH_INFO: `/v3/services/image` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:42.868875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f924ba-4bc8-4eae-a26e-31d11597eecf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:42.871430 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f924ba-4bc8-4eae-a26e-31d11597eecf None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-e4f924ba-4bc8-4eae-a26e-31d11597eecf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:42.888624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:42.889660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:42.890534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 513/989] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Oct 4 11:41:42 2022] GET /identity/v3/services/image => generated 86 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:42.901048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f924ba-4bc8-4eae-a26e-31d11597eecf None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:42.977256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694e39d-cd96-4324-8187-326372a963f7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:42.988949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:43.024943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8caebf-4fdb-401d-b79f-d204fcb44199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:43.025589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8caebf-4fdb-401d-b79f-d204fcb44199 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:43.025866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8caebf-4fdb-401d-b79f-d204fcb44199 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:43.026139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8caebf-4fdb-401d-b79f-d204fcb44199 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:43.026904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8caebf-4fdb-401d-b79f-d204fcb44199 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:43.027242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8caebf-4fdb-401d-b79f-d204fcb44199 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:43.028758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8caebf-4fdb-401d-b79f-d204fcb44199 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:43.036764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 477/990] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 11:41:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:43.042653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8caebf-4fdb-401d-b79f-d204fcb44199 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:43.062734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f924ba-4bc8-4eae-a26e-31d11597eecf None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:43.110206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dced2a-823d-43fa-983e-6a9e2772f1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:43.110822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dced2a-823d-43fa-983e-6a9e2772f1e6 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:43.111234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dced2a-823d-43fa-983e-6a9e2772f1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:43.111399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dced2a-823d-43fa-983e-6a9e2772f1e6 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:43.111998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dced2a-823d-43fa-983e-6a9e2772f1e6 None admin] RBAC: Adding query filter params (name=image) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:43.112335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dced2a-823d-43fa-983e-6a9e2772f1e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:43.113748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dced2a-823d-43fa-983e-6a9e2772f1e6 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:43.124758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 514/991] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:43.127168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dced2a-823d-43fa-983e-6a9e2772f1e6 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:43.159714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:43.173603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8caebf-4fdb-401d-b79f-d204fcb44199 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:43.219718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd81abc-27ce-4dc7-a171-b6a68f3039f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:43.220364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd81abc-27ce-4dc7-a171-b6a68f3039f0 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:43.221003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd81abc-27ce-4dc7-a171-b6a68f3039f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:43.221003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd81abc-27ce-4dc7-a171-b6a68f3039f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:43.221618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd81abc-27ce-4dc7-a171-b6a68f3039f0 None admin] RBAC: Adding query filter params (interface=admin, service_id=30e0a83e72564e81a449524728f04ae5, region_id=RegionOne) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:43.221922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd81abc-27ce-4dc7-a171-b6a68f3039f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:43.223343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd81abc-27ce-4dc7-a171-b6a68f3039f0 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:43.230014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 478/992] 10.222.0.15 () {56 vars in 1249 bytes} [Tue Oct 4 11:41:43 2022] GET /identity/v3/endpoints?service_id=30e0a83e72564e81a449524728f04ae5&interface=admin®ion_id=RegionOne => generated 193 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:43.236614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd81abc-27ce-4dc7-a171-b6a68f3039f0 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:43.292741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dced2a-823d-43fa-983e-6a9e2772f1e6 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:43.306048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:43.347040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fdc5a-57c0-4db1-977d-06750bf049bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:43.347656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fdc5a-57c0-4db1-977d-06750bf049bb None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:43.347935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fdc5a-57c0-4db1-977d-06750bf049bb None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:43.348236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fdc5a-57c0-4db1-977d-06750bf049bb None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:43.348893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fdc5a-57c0-4db1-977d-06750bf049bb None admin] RBAC: Adding query filter params (type=image) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:43.349194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fdc5a-57c0-4db1-977d-06750bf049bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:43.350601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fdc5a-57c0-4db1-977d-06750bf049bb None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:43.356821 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 515/993] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:43 2022] GET /identity/v3/services?type=image => generated 122 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:43.393301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd81abc-27ce-4dc7-a171-b6a68f3039f0 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:43.441894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda9a2a-1cac-4530-8fb3-ede19eb73b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:43.443012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda9a2a-1cac-4530-8fb3-ede19eb73b26 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:43.443587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda9a2a-1cac-4530-8fb3-ede19eb73b26 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:43.444080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda9a2a-1cac-4530-8fb3-ede19eb73b26 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:43.445083 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda9a2a-1cac-4530-8fb3-ede19eb73b26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:43.446885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda9a2a-1cac-4530-8fb3-ede19eb73b26 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:43.460255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 479/994] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 11:41:43 2022] GET /identity/v3/services => generated 1727 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 11:41:43.555491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f367f-72fe-479c-bdcc-fcb0fd843cbd None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:43.555859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f367f-72fe-479c-bdcc-fcb0fd843cbd None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:43.556051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f367f-72fe-479c-bdcc-fcb0fd843cbd None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:43.557488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 516/995] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:43.564322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372620f-9edc-4cf6-8a33-7bf17968c2d1 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:43.564618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372620f-9edc-4cf6-8a33-7bf17968c2d1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:43.564881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372620f-9edc-4cf6-8a33-7bf17968c2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:43.596200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a372620f-9edc-4cf6-8a33-7bf17968c2d1 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:43.596200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a372620f-9edc-4cf6-8a33-7bf17968c2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:43.704319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 480/996] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:44.030387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f367f-72fe-479c-bdcc-fcb0fd843cbd None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:44.149226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:44.181550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f367f-72fe-479c-bdcc-fcb0fd843cbd None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:44.225144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb31ccd-e613-474f-a448-a857602bd976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:44.225802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb31ccd-e613-474f-a448-a857602bd976 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:44.226141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb31ccd-e613-474f-a448-a857602bd976 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:44.226483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb31ccd-e613-474f-a448-a857602bd976 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:44.227090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb31ccd-e613-474f-a448-a857602bd976 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:44.228537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb31ccd-e613-474f-a448-a857602bd976 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-2fb31ccd-e613-474f-a448-a857602bd976 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:44.238954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:44.244877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:44.245763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 517/997] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:44.251097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372620f-9edc-4cf6-8a33-7bf17968c2d1 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:44.308693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:44.410884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372620f-9edc-4cf6-8a33-7bf17968c2d1 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:44.460948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e0751-5b4c-423f-9e2d-9502e45e8ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:44.461633 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e0751-5b4c-423f-9e2d-9502e45e8ada None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:44.462149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e0751-5b4c-423f-9e2d-9502e45e8ada None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:44.462536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e0751-5b4c-423f-9e2d-9502e45e8ada None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:44.463316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e0751-5b4c-423f-9e2d-9502e45e8ada None admin] RBAC: Adding query filter params (name=compute) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:44.463673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e0751-5b4c-423f-9e2d-9502e45e8ada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:44.465475 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e0751-5b4c-423f-9e2d-9502e45e8ada None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:44.482267 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 481/998] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:44.482772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb31ccd-e613-474f-a448-a857602bd976 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:44.658091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb31ccd-e613-474f-a448-a857602bd976 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:44.720676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a282c-6dda-469a-b091-1c6f0ae2fc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:44.721767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a282c-6dda-469a-b091-1c6f0ae2fc37 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:44.722236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a282c-6dda-469a-b091-1c6f0ae2fc37 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:44.722707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a282c-6dda-469a-b091-1c6f0ae2fc37 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:44.723814 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a282c-6dda-469a-b091-1c6f0ae2fc37 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:44.724329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a282c-6dda-469a-b091-1c6f0ae2fc37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:44.727420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a282c-6dda-469a-b091-1c6f0ae2fc37 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:44.737278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 518/999] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:44.746751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e0751-5b4c-423f-9e2d-9502e45e8ada None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:44.913363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:44.925979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e0751-5b4c-423f-9e2d-9502e45e8ada None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:44.979340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291bf7b-eee7-42ff-92a6-b0fdaea2a9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:44.980207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291bf7b-eee7-42ff-92a6-b0fdaea2a9e2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:44.980560 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291bf7b-eee7-42ff-92a6-b0fdaea2a9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:44.980998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291bf7b-eee7-42ff-92a6-b0fdaea2a9e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:44.981809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291bf7b-eee7-42ff-92a6-b0fdaea2a9e2 None admin] RBAC: Adding query filter params (interface=public, service_id=cdc891e403324bbc8e199e650213638c, region_id=RegionOne) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:44.982132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291bf7b-eee7-42ff-92a6-b0fdaea2a9e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:44.983867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291bf7b-eee7-42ff-92a6-b0fdaea2a9e2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:44.991662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 482/1000] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 11:41:44 2022] GET /identity/v3/endpoints?service_id=cdc891e403324bbc8e199e650213638c&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:44.999604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a282c-6dda-469a-b091-1c6f0ae2fc37 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:45.020086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:45.181199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a282c-6dda-469a-b091-1c6f0ae2fc37 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:45.245583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb89e83-fded-4552-87d9-fb9d312e6d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:45.246575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb89e83-fded-4552-87d9-fb9d312e6d3d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:45.247032 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb89e83-fded-4552-87d9-fb9d312e6d3d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:45.247489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb89e83-fded-4552-87d9-fb9d312e6d3d None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:45.248561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb89e83-fded-4552-87d9-fb9d312e6d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:45.251162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb89e83-fded-4552-87d9-fb9d312e6d3d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:45.264453 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 519/1001] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 11:41:44 2022] GET /identity/v3/services => generated 1727 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 11:41:45.719535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85426a-a4bb-4527-962b-3a9f8ca720eb None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:45.720099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85426a-a4bb-4527-962b-3a9f8ca720eb None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:45.720099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85426a-a4bb-4527-962b-3a9f8ca720eb None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:45.721611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 483/1002] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:45.727242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a475aaf-8050-4b1e-8698-425b60b0b5c4 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:45.727594 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a475aaf-8050-4b1e-8698-425b60b0b5c4 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:45.727793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a475aaf-8050-4b1e-8698-425b60b0b5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:45.749625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-8a475aaf-8050-4b1e-8698-425b60b0b5c4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:45.750950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8a475aaf-8050-4b1e-8698-425b60b0b5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:45.808640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd473-9ae6-474f-98ae-ae47149f572f None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:45.809077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd473-9ae6-474f-98ae-ae47149f572f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:45.809521 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebd473-9ae6-474f-98ae-ae47149f572f None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:45.812439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 484/1003] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:45.818991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36dcad-8c48-4e57-9e70-bc777430c750 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:45.819446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36dcad-8c48-4e57-9e70-bc777430c750 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:45.819951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36dcad-8c48-4e57-9e70-bc777430c750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:45.847909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-af36dcad-8c48-4e57-9e70-bc777430c750 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:45.849923 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-af36dcad-8c48-4e57-9e70-bc777430c750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:45.894422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 520/1004] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:45.979965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 485/1005] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:46.166983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a475aaf-8050-4b1e-8698-425b60b0b5c4 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:46.169467 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:46.280634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a475aaf-8050-4b1e-8698-425b60b0b5c4 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:46.319896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0fca9-388c-4153-9978-9cebe1d4ab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:46.320537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0fca9-388c-4153-9978-9cebe1d4ab6d None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:46.320811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0fca9-388c-4153-9978-9cebe1d4ab6d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:46.321080 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0fca9-388c-4153-9978-9cebe1d4ab6d None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:46.321744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0fca9-388c-4153-9978-9cebe1d4ab6d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:46.323122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0fca9-388c-4153-9978-9cebe1d4ab6d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:46.425806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 521/1006] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:41:46 2022] POST /identity/v3/services => generated 242 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:41:46.489896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36dcad-8c48-4e57-9e70-bc777430c750 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:46.609163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:46.624528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36dcad-8c48-4e57-9e70-bc777430c750 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:46.671004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536f904-74af-4f6a-976a-257e90fd185b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:46.671649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536f904-74af-4f6a-976a-257e90fd185b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:46.671961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536f904-74af-4f6a-976a-257e90fd185b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:46.672279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536f904-74af-4f6a-976a-257e90fd185b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:46.672825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536f904-74af-4f6a-976a-257e90fd185b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:46.674225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536f904-74af-4f6a-976a-257e90fd185b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-a536f904-74af-4f6a-976a-257e90fd185b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:46.690232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:46.691376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 11:41:46.692272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:46.693205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 486/1007] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Oct 4 11:41:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:46.703667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0fca9-388c-4153-9978-9cebe1d4ab6d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:46.817548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:46.894477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0fca9-388c-4153-9978-9cebe1d4ab6d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:46.962140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621c441-019a-4374-b5d5-a9169a6dd5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:46.963205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621c441-019a-4374-b5d5-a9169a6dd5c1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:46.963682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621c441-019a-4374-b5d5-a9169a6dd5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:46.964143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621c441-019a-4374-b5d5-a9169a6dd5c1 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:46.965290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621c441-019a-4374-b5d5-a9169a6dd5c1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:46.965849 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621c441-019a-4374-b5d5-a9169a6dd5c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:46.968332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621c441-019a-4374-b5d5-a9169a6dd5c1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:46.984336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 522/1008] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 11:41:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:46.988639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536f904-74af-4f6a-976a-257e90fd185b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:47.159427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536f904-74af-4f6a-976a-257e90fd185b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:47.196806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f041804-1e9c-442f-90d3-8611d5c78104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:47.197421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f041804-1e9c-442f-90d3-8611d5c78104 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:47.197699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f041804-1e9c-442f-90d3-8611d5c78104 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:47.197968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f041804-1e9c-442f-90d3-8611d5c78104 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:47.198604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f041804-1e9c-442f-90d3-8611d5c78104 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:47.198910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f041804-1e9c-442f-90d3-8611d5c78104 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:47.200368 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f041804-1e9c-442f-90d3-8611d5c78104 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:47.206290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 487/1009] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Oct 4 11:41:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:47.213601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621c441-019a-4374-b5d5-a9169a6dd5c1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:47.393442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621c441-019a-4374-b5d5-a9169a6dd5c1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:47.426662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:47.439952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff8008-2b03-4007-815f-b3b127d8b72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:47.440535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff8008-2b03-4007-815f-b3b127d8b72a None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:47.440818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff8008-2b03-4007-815f-b3b127d8b72a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:47.441081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff8008-2b03-4007-815f-b3b127d8b72a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:47.442006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff8008-2b03-4007-815f-b3b127d8b72a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:47.443448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff8008-2b03-4007-815f-b3b127d8b72a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:47.521245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 523/1010] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 11:41:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:41:48.065617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310c08a-d492-4f80-a8f4-f543c1a33343 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:48.066348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310c08a-d492-4f80-a8f4-f543c1a33343 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:48.066348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310c08a-d492-4f80-a8f4-f543c1a33343 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:48.067590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 488/1011] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:48.073611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b096-765b-46ab-b6f8-0485b9fe297b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:48.074007 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b096-765b-46ab-b6f8-0485b9fe297b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:48.074286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b096-765b-46ab-b6f8-0485b9fe297b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:48.098360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-7835b096-765b-46ab-b6f8-0485b9fe297b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:48.100387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7835b096-765b-46ab-b6f8-0485b9fe297b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:48.239325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 524/1012] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:48 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:48.596086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310c08a-d492-4f80-a8f4-f543c1a33343 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:48.613910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:48.793491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310c08a-d492-4f80-a8f4-f543c1a33343 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:48.861919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55797e-32b2-4490-a81f-22419b2258d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:48.862911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55797e-32b2-4490-a81f-22419b2258d2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:48.863416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55797e-32b2-4490-a81f-22419b2258d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:48.863921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55797e-32b2-4490-a81f-22419b2258d2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:48.865042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55797e-32b2-4490-a81f-22419b2258d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:48.867669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55797e-32b2-4490-a81f-22419b2258d2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-4a55797e-32b2-4490-a81f-22419b2258d2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:48.889079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:48.891108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:48.899630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 489/1013] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:48.904565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835b096-765b-46ab-b6f8-0485b9fe297b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:49.094227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835b096-765b-46ab-b6f8-0485b9fe297b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:49.121253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:49.158485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3dff2-bc82-4733-a154-be1fc38a141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:49.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3dff2-bc82-4733-a154-be1fc38a141e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:49.160767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3dff2-bc82-4733-a154-be1fc38a141e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:49.161793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3dff2-bc82-4733-a154-be1fc38a141e None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:49.163284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3dff2-bc82-4733-a154-be1fc38a141e None admin] RBAC: Adding query filter params (name=compute) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:49.165455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3dff2-bc82-4733-a154-be1fc38a141e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:49.168101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3dff2-bc82-4733-a154-be1fc38a141e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:49.184203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 525/1014] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:49.188952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55797e-32b2-4490-a81f-22419b2258d2 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:49.332645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:49.384127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55797e-32b2-4490-a81f-22419b2258d2 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:49.424521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0a81-e2db-457b-be03-b60dc9e286e9 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:49.424968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0a81-e2db-457b-be03-b60dc9e286e9 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:49.425415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0a81-e2db-457b-be03-b60dc9e286e9 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:49.427876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 526/1015] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:49.437442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d16ec-8f28-4979-8e43-ffad60894291 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:49.437812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d16ec-8f28-4979-8e43-ffad60894291 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:49.438257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d16ec-8f28-4979-8e43-ffad60894291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:49.444794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d4b40-5d22-4d8b-8383-7e7dd7bee292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:49.445743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d4b40-5d22-4d8b-8383-7e7dd7bee292 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:49.446187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d4b40-5d22-4d8b-8383-7e7dd7bee292 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:49.446615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d4b40-5d22-4d8b-8383-7e7dd7bee292 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:49.447709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d4b40-5d22-4d8b-8383-7e7dd7bee292 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:49.448152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d4b40-5d22-4d8b-8383-7e7dd7bee292 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:49.450532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d4b40-5d22-4d8b-8383-7e7dd7bee292 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:49.464476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 490/1016] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Oct 4 11:41:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:49.465505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c51d16ec-8f28-4979-8e43-ffad60894291 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:49.466427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d4b40-5d22-4d8b-8383-7e7dd7bee292 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:49.467016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c51d16ec-8f28-4979-8e43-ffad60894291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:49.547130 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 527/1017] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:49 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:49.594558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d4b40-5d22-4d8b-8383-7e7dd7bee292 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:49.643373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af3e3-a1ac-4435-9c6a-07c5ca8148b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:49.644053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af3e3-a1ac-4435-9c6a-07c5ca8148b2 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:49.644444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af3e3-a1ac-4435-9c6a-07c5ca8148b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:49.644715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af3e3-a1ac-4435-9c6a-07c5ca8148b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:49.645416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47af3e3-a1ac-4435-9c6a-07c5ca8148b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:49.646826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47af3e3-a1ac-4435-9c6a-07c5ca8148b2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:49.658927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:49.761475 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 491/1018] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 11:41:49 2022] POST /identity/v3/endpoints => generated 334 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:41:49.883679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d16ec-8f28-4979-8e43-ffad60894291 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:49.912294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82e24b-90eb-4f93-a62b-71ebdce485a3 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:49.913037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82e24b-90eb-4f93-a62b-71ebdce485a3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:49.913315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82e24b-90eb-4f93-a62b-71ebdce485a3 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:49.915456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 492/1019] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:49.922514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6042c-f56e-406c-954c-34cfea3a5d78 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:49.922795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6042c-f56e-406c-954c-34cfea3a5d78 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:49.923054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6042c-f56e-406c-954c-34cfea3a5d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:49.946357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-c0f6042c-f56e-406c-954c-34cfea3a5d78 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:49.948277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c0f6042c-f56e-406c-954c-34cfea3a5d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:50.029018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d16ec-8f28-4979-8e43-ffad60894291 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:50.040961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:50.047819 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 493/1020] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:50.089558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859c394-37fe-41f0-907c-ab5ef50ac26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:50.090548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859c394-37fe-41f0-907c-ab5ef50ac26c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:50.091017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859c394-37fe-41f0-907c-ab5ef50ac26c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:50.091432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859c394-37fe-41f0-907c-ab5ef50ac26c None admin] PATH_INFO: `/v3/services/image` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:50.092438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859c394-37fe-41f0-907c-ab5ef50ac26c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:50.094812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859c394-37fe-41f0-907c-ab5ef50ac26c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-7859c394-37fe-41f0-907c-ab5ef50ac26c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:50.110758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:50.111819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 11:41:50.112717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:50.113466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 528/1021] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Oct 4 11:41:49 2022] GET /identity/v3/services/image => generated 86 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:50.121720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6042c-f56e-406c-954c-34cfea3a5d78 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:50.255493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6042c-f56e-406c-954c-34cfea3a5d78 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:50.280783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:50.293244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec253406-2e58-4493-8b78-a581b3e3870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:50.293856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec253406-2e58-4493-8b78-a581b3e3870a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:50.294135 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec253406-2e58-4493-8b78-a581b3e3870a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:50.294407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec253406-2e58-4493-8b78-a581b3e3870a None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:50.295211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec253406-2e58-4493-8b78-a581b3e3870a None admin] RBAC: Adding query filter params (name=image) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:50.295513 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec253406-2e58-4493-8b78-a581b3e3870a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:50.296922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec253406-2e58-4493-8b78-a581b3e3870a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:50.302549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 494/1022] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:50.309304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859c394-37fe-41f0-907c-ab5ef50ac26c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:50.322664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec253406-2e58-4493-8b78-a581b3e3870a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:50.441195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859c394-37fe-41f0-907c-ab5ef50ac26c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:50.441402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec253406-2e58-4493-8b78-a581b3e3870a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:50.482828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d04d5-9207-4011-863f-18309a28da56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:50.483875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d04d5-9207-4011-863f-18309a28da56 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:50.484145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d04d5-9207-4011-863f-18309a28da56 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:50.484355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d04d5-9207-4011-863f-18309a28da56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:50.485487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ec1b2-a9a0-487a-b10b-1d055337b893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:50.485634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d04d5-9207-4011-863f-18309a28da56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:50.485945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ec1b2-a9a0-487a-b10b-1d055337b893 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:50.486385 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ec1b2-a9a0-487a-b10b-1d055337b893 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:50.486548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ec1b2-a9a0-487a-b10b-1d055337b893 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:50.487102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d04d5-9207-4011-863f-18309a28da56 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:50.487209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ec1b2-a9a0-487a-b10b-1d055337b893 None admin] RBAC: Adding query filter params (type=image) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:50.487482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ec1b2-a9a0-487a-b10b-1d055337b893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:50.489627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 495/1023] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:50.492581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ec1b2-a9a0-487a-b10b-1d055337b893 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:50.495944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d04d5-9207-4011-863f-18309a28da56 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:50.503648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 529/1024] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:50.510167 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ec1b2-a9a0-487a-b10b-1d055337b893 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:50.546101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:50.573085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:50.623195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d04d5-9207-4011-863f-18309a28da56 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:50.623398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ec1b2-a9a0-487a-b10b-1d055337b893 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:50.671993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd415306-ad93-48fc-bb23-4ad7d6cba64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:50.672650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e499d32-513e-49ce-8768-d697533ef449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:50.672786 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd415306-ad93-48fc-bb23-4ad7d6cba64f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:50.672866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e499d32-513e-49ce-8768-d697533ef449 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:50.673132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd415306-ad93-48fc-bb23-4ad7d6cba64f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:50.673232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e499d32-513e-49ce-8768-d697533ef449 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:50.673305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd415306-ad93-48fc-bb23-4ad7d6cba64f None admin] PATH_INFO: `/v3/endpoints` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:50.673384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e499d32-513e-49ce-8768-d697533ef449 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:50.673935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd415306-ad93-48fc-bb23-4ad7d6cba64f None admin] RBAC: Adding query filter params (interface=public, service_id=8d515bfff1934d84bde84fa6706daa64, region_id=RegionOne) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:50.674039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e499d32-513e-49ce-8768-d697533ef449 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:50.674300 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd415306-ad93-48fc-bb23-4ad7d6cba64f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:50.675506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e499d32-513e-49ce-8768-d697533ef449 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:50.675946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd415306-ad93-48fc-bb23-4ad7d6cba64f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:50.700205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 530/1025] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 11:41:50 2022] GET /identity/v3/endpoints?service_id=8d515bfff1934d84bde84fa6706daa64&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:50.703456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd415306-ad93-48fc-bb23-4ad7d6cba64f None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:50.809206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 496/1026] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:41:50 2022] POST /identity/v3/projects => generated 311 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 11:41:50.815561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:50.821227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd415306-ad93-48fc-bb23-4ad7d6cba64f None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:50.862693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a028b-2b0f-4f87-967a-726b5e68e261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:50.863435 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a028b-2b0f-4f87-967a-726b5e68e261 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:50.863728 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a028b-2b0f-4f87-967a-726b5e68e261 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:50.864044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a028b-2b0f-4f87-967a-726b5e68e261 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:50.864739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a028b-2b0f-4f87-967a-726b5e68e261 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:50.866241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a028b-2b0f-4f87-967a-726b5e68e261 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:50.875716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 531/1027] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 11:41:50 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 11:41:52.171035 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27aebfa-c5ad-443c-ba4f-a190aa0d8cc3 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:52.171741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27aebfa-c5ad-443c-ba4f-a190aa0d8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:52.171741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27aebfa-c5ad-443c-ba4f-a190aa0d8cc3 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:52.173016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 497/1028] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:52.178642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71dd22-8e09-467c-a89a-db4b18e1211d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:52.179684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71dd22-8e09-467c-a89a-db4b18e1211d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:52.179810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71dd22-8e09-467c-a89a-db4b18e1211d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:52.198732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1a71dd22-8e09-467c-a89a-db4b18e1211d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:52.199929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1a71dd22-8e09-467c-a89a-db4b18e1211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:52.284043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 532/1029] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:52.563984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27aebfa-c5ad-443c-ba4f-a190aa0d8cc3 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:52.678124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27aebfa-c5ad-443c-ba4f-a190aa0d8cc3 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:52.686467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:52.721615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d46bb6-7f41-4e06-96bb-4f958c9a98a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:52.722251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d46bb6-7f41-4e06-96bb-4f958c9a98a4 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:52.722526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d46bb6-7f41-4e06-96bb-4f958c9a98a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:52.722837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d46bb6-7f41-4e06-96bb-4f958c9a98a4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:52.734253 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d46bb6-7f41-4e06-96bb-4f958c9a98a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-38d46bb6-7f41-4e06-96bb-4f958c9a98a4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:52.739126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:52.740147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:52.741107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 498/1030] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 11:41:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:52.746458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71dd22-8e09-467c-a89a-db4b18e1211d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:52.858321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:52.867720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71dd22-8e09-467c-a89a-db4b18e1211d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:52.906295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba810ff1-024f-4617-a97f-7963e535aae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:52.906927 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba810ff1-024f-4617-a97f-7963e535aae6 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:52.907210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba810ff1-024f-4617-a97f-7963e535aae6 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:52.907485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba810ff1-024f-4617-a97f-7963e535aae6 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:52.908195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba810ff1-024f-4617-a97f-7963e535aae6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:52.908522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba810ff1-024f-4617-a97f-7963e535aae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:52.909975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba810ff1-024f-4617-a97f-7963e535aae6 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:52.924649 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 533/1031] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 11:41:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:52.926415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d46bb6-7f41-4e06-96bb-4f958c9a98a4 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:53.054973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d46bb6-7f41-4e06-96bb-4f958c9a98a4 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:53.073339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0121d52-541c-4179-9b48-0b07fac1aabb None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.073632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0121d52-541c-4179-9b48-0b07fac1aabb None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.073903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0121d52-541c-4179-9b48-0b07fac1aabb None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.075542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 534/1032] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:53.081513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cab199-97d3-4ae8-bb94-e880b3856b24 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.081795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cab199-97d3-4ae8-bb94-e880b3856b24 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.082069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cab199-97d3-4ae8-bb94-e880b3856b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.095454 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559d391-5a77-499b-a9f2-0f4fc3b07e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:53.096412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559d391-5a77-499b-a9f2-0f4fc3b07e5d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.096903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559d391-5a77-499b-a9f2-0f4fc3b07e5d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.097330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559d391-5a77-499b-a9f2-0f4fc3b07e5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.107388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-09cab199-97d3-4ae8-bb94-e880b3856b24 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:53.108950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-09cab199-97d3-4ae8-bb94-e880b3856b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:53.109681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559d391-5a77-499b-a9f2-0f4fc3b07e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:53.111308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559d391-5a77-499b-a9f2-0f4fc3b07e5d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-0559d391-5a77-499b-a9f2-0f4fc3b07e5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:53.126879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:53.128205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:53.129332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:53.129332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:53.129332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:53.130748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 499/1033] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:53.140423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559d391-5a77-499b-a9f2-0f4fc3b07e5d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:53.161988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:53.187339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 535/1034] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:53.188744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59182fd9-b8d4-43ea-9be3-67586ed36073 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.189103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59182fd9-b8d4-43ea-9be3-67586ed36073 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.189396 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59182fd9-b8d4-43ea-9be3-67586ed36073 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.195776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 536/1035] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:53.197239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e408f-7cf2-4d37-9564-bf06b7050a39 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.197539 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e408f-7cf2-4d37-9564-bf06b7050a39 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.197850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e408f-7cf2-4d37-9564-bf06b7050a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.219854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f58e408f-7cf2-4d37-9564-bf06b7050a39 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:53.221177 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f58e408f-7cf2-4d37-9564-bf06b7050a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:53.263310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559d391-5a77-499b-a9f2-0f4fc3b07e5d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:53.316209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 537/1036] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:53.320279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a926958-0f08-4537-82cc-34afbe54acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:53.320980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a926958-0f08-4537-82cc-34afbe54acf6 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.321327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a926958-0f08-4537-82cc-34afbe54acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.321678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a926958-0f08-4537-82cc-34afbe54acf6 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.322457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a926958-0f08-4537-82cc-34afbe54acf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:53.322835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a926958-0f08-4537-82cc-34afbe54acf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:53.324356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a926958-0f08-4537-82cc-34afbe54acf6 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:53.341340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 500/1037] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:53.346996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e408f-7cf2-4d37-9564-bf06b7050a39 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:53.364152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:53.453075 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e408f-7cf2-4d37-9564-bf06b7050a39 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:53.459508 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a926958-0f08-4537-82cc-34afbe54acf6 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:53.492005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f796cb2-3326-4d4c-80fa-17af939abf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:53.492690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f796cb2-3326-4d4c-80fa-17af939abf91 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.492974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f796cb2-3326-4d4c-80fa-17af939abf91 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.493246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f796cb2-3326-4d4c-80fa-17af939abf91 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.501322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f796cb2-3326-4d4c-80fa-17af939abf91 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:53.502808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f796cb2-3326-4d4c-80fa-17af939abf91 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-9f796cb2-3326-4d4c-80fa-17af939abf91 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:53.518981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:53.520085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:53.521141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:53.521141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:53.521141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:53.521141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:53.521141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:53.521141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:53.521141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:53.521141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:53.521141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:41:53.521141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:53.526397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 538/1038] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 11:41:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:53.527338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f796cb2-3326-4d4c-80fa-17af939abf91 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:53.563458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:53.588711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a926958-0f08-4537-82cc-34afbe54acf6 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:53.626108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb7ce7-1f20-4dc5-b58f-6187c204bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:53.626314 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:53.626870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7ce7-1f20-4dc5-b58f-6187c204bab7 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.627146 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7ce7-1f20-4dc5-b58f-6187c204bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.627520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7ce7-1f20-4dc5-b58f-6187c204bab7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.628979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb7ce7-1f20-4dc5-b58f-6187c204bab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:53.630825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb7ce7-1f20-4dc5-b58f-6187c204bab7 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:53.633396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 501/1039] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:53.638688 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb7ce7-1f20-4dc5-b58f-6187c204bab7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:53.656566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f796cb2-3326-4d4c-80fa-17af939abf91 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:53.693575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f057a92-ce99-4e95-8bba-eda5e4051cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:53.694246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f057a92-ce99-4e95-8bba-eda5e4051cc9 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.694471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f057a92-ce99-4e95-8bba-eda5e4051cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.695238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f057a92-ce99-4e95-8bba-eda5e4051cc9 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.695975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f057a92-ce99-4e95-8bba-eda5e4051cc9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:53.696315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f057a92-ce99-4e95-8bba-eda5e4051cc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:53.697761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f057a92-ce99-4e95-8bba-eda5e4051cc9 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:53.716248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 539/1040] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 11:41:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:53.717079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f057a92-ce99-4e95-8bba-eda5e4051cc9 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:53.746603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb7ce7-1f20-4dc5-b58f-6187c204bab7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:53.793968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff71e45-59b5-4da7-ae09-5fa9fb16f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:53.794635 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff71e45-59b5-4da7-ae09-5fa9fb16f1e0 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.794915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff71e45-59b5-4da7-ae09-5fa9fb16f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.795186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff71e45-59b5-4da7-ae09-5fa9fb16f1e0 None admin] PATH_INFO: `/v3/services/image` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.795768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff71e45-59b5-4da7-ae09-5fa9fb16f1e0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:53.797210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff71e45-59b5-4da7-ae09-5fa9fb16f1e0 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-2ff71e45-59b5-4da7-ae09-5fa9fb16f1e0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:53.807678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:53.812608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:53.813615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 502/1041] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Oct 4 11:41:53 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:53.814607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff71e45-59b5-4da7-ae09-5fa9fb16f1e0 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:53.824112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:53.838964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f057a92-ce99-4e95-8bba-eda5e4051cc9 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:53.870592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:53.876624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e967bf7-6590-4bac-be64-12893eb495f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:53.877265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e967bf7-6590-4bac-be64-12893eb495f3 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.877549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e967bf7-6590-4bac-be64-12893eb495f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.877890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e967bf7-6590-4bac-be64-12893eb495f3 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.878557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e967bf7-6590-4bac-be64-12893eb495f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:53.880000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e967bf7-6590-4bac-be64-12893eb495f3 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:53.923764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff71e45-59b5-4da7-ae09-5fa9fb16f1e0 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:53.958889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef7cdf-7530-48b8-a6dc-963fe7e75ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:53.959489 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7cdf-7530-48b8-a6dc-963fe7e75ee2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:53.959767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7cdf-7530-48b8-a6dc-963fe7e75ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:53.960032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7cdf-7530-48b8-a6dc-963fe7e75ee2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:53.988209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 540/1042] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:41:53 2022] POST /identity/v3/users => generated 285 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:53.988209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e967bf7-6590-4bac-be64-12893eb495f3 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:54.037448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef7cdf-7530-48b8-a6dc-963fe7e75ee2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:54.038970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef7cdf-7530-48b8-a6dc-963fe7e75ee2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:54.052684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:54.140628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e967bf7-6590-4bac-be64-12893eb495f3 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:54.165403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 503/1043] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:53 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:41:54.171772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef7cdf-7530-48b8-a6dc-963fe7e75ee2 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:54.197540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d6d37-9790-4d96-a2db-0e9f06dcd0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:54.198645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6d37-9790-4d96-a2db-0e9f06dcd0ea None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:54.199971 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6d37-9790-4d96-a2db-0e9f06dcd0ea None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:54.199971 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6d37-9790-4d96-a2db-0e9f06dcd0ea None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:54.200970 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d6d37-9790-4d96-a2db-0e9f06dcd0ea None admin] RBAC: Adding query filter params (name=image) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:54.201542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d6d37-9790-4d96-a2db-0e9f06dcd0ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:54.204332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d6d37-9790-4d96-a2db-0e9f06dcd0ea None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:54.213969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 541/1044] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:54.222003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d6d37-9790-4d96-a2db-0e9f06dcd0ea None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:54.293811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:54.308359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef7cdf-7530-48b8-a6dc-963fe7e75ee2 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:54.321621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:54.347837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03162e2e-fccb-4a52-82aa-001e6fe6c454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:54.348533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03162e2e-fccb-4a52-82aa-001e6fe6c454 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:54.348920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03162e2e-fccb-4a52-82aa-001e6fe6c454 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:54.349264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03162e2e-fccb-4a52-82aa-001e6fe6c454 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:54.353464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d6d37-9790-4d96-a2db-0e9f06dcd0ea None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:54.362147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03162e2e-fccb-4a52-82aa-001e6fe6c454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:54.363645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03162e2e-fccb-4a52-82aa-001e6fe6c454 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-03162e2e-fccb-4a52-82aa-001e6fe6c454 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:54.386477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:54.387614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:54.388627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:54.388627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:54.388627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:54.388627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 504/1045] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:54.393759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03162e2e-fccb-4a52-82aa-001e6fe6c454 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:54.400157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3c904-5ffd-4bc6-8e5b-42ba9494c572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:54.401123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3c904-5ffd-4bc6-8e5b-42ba9494c572 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:54.401495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3c904-5ffd-4bc6-8e5b-42ba9494c572 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:54.401768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3c904-5ffd-4bc6-8e5b-42ba9494c572 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:54.402407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3c904-5ffd-4bc6-8e5b-42ba9494c572 None admin] RBAC: Adding query filter params (type=image) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:54.402708 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3c904-5ffd-4bc6-8e5b-42ba9494c572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:54.404090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3c904-5ffd-4bc6-8e5b-42ba9494c572 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:54.409530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 542/1046] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:54.416792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3c904-5ffd-4bc6-8e5b-42ba9494c572 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:54.529165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03162e2e-fccb-4a52-82aa-001e6fe6c454 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:54.563481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3c904-5ffd-4bc6-8e5b-42ba9494c572 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:54.586402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:54.588217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93439df4-0925-4b7d-9cf8-489d714cacd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:54.588841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439df4-0925-4b7d-9cf8-489d714cacd2 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:54.589126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439df4-0925-4b7d-9cf8-489d714cacd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:54.589402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439df4-0925-4b7d-9cf8-489d714cacd2 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:54.590056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93439df4-0925-4b7d-9cf8-489d714cacd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:54.590364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93439df4-0925-4b7d-9cf8-489d714cacd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:54.592006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93439df4-0925-4b7d-9cf8-489d714cacd2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:54.604989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 505/1047] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:54.610817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93439df4-0925-4b7d-9cf8-489d714cacd2 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:54.614027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0696b98-de3a-4582-9648-e7ce54ff70d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:54.615974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0696b98-de3a-4582-9648-e7ce54ff70d6 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:54.616654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0696b98-de3a-4582-9648-e7ce54ff70d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:54.617111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0696b98-de3a-4582-9648-e7ce54ff70d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:54.618041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0696b98-de3a-4582-9648-e7ce54ff70d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:54.619647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0696b98-de3a-4582-9648-e7ce54ff70d6 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:54.640726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:54.716294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 543/1048] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 11:41:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:41:54.730383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93439df4-0925-4b7d-9cf8-489d714cacd2 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:54.782217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381394d1-4575-4927-adec-9a74e99eb158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:54.786069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381394d1-4575-4927-adec-9a74e99eb158 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:54.786069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381394d1-4575-4927-adec-9a74e99eb158 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:54.786069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381394d1-4575-4927-adec-9a74e99eb158 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:54.800111 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381394d1-4575-4927-adec-9a74e99eb158 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:54.802108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381394d1-4575-4927-adec-9a74e99eb158 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-381394d1-4575-4927-adec-9a74e99eb158 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:54.816531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:54.817732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:54.818765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:54.818765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:54.818765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:54.818765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:54.818765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:54.818765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:54.818765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:54.818765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:54.818765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 506/1049] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:54.823772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0696b98-de3a-4582-9648-e7ce54ff70d6 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:54.940827 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0696b98-de3a-4582-9648-e7ce54ff70d6 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:54.979210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f475f46-c940-415d-8486-c5c479bfbc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:54.980244 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475f46-c940-415d-8486-c5c479bfbc5d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:54.980642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475f46-c940-415d-8486-c5c479bfbc5d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:54.981111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475f46-c940-415d-8486-c5c479bfbc5d None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:54.982337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f475f46-c940-415d-8486-c5c479bfbc5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:54.983101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f475f46-c940-415d-8486-c5c479bfbc5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:54.985753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f475f46-c940-415d-8486-c5c479bfbc5d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:55.006806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 544/1050] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:55.013671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381394d1-4575-4927-adec-9a74e99eb158 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:55.100510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:55.123775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381394d1-4575-4927-adec-9a74e99eb158 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:55.163622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6487c5-c3e8-40b7-af84-29a990d73553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:55.164254 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6487c5-c3e8-40b7-af84-29a990d73553 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:55.164569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6487c5-c3e8-40b7-af84-29a990d73553 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:55.164802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6487c5-c3e8-40b7-af84-29a990d73553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:55.165533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6487c5-c3e8-40b7-af84-29a990d73553 None admin] RBAC: Adding query filter params (role.id=87a27c6c32d94ae7b373df89e15cb25e, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=ed14c1bc105d4c5cade5813edc1f0dac) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:55.165845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6487c5-c3e8-40b7-af84-29a990d73553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:55.167281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6487c5-c3e8-40b7-af84-29a990d73553 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:55.173309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 507/1051] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:41:55 2022] GET /identity/v3/role_assignments?user.id=ed14c1bc105d4c5cade5813edc1f0dac&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=87a27c6c32d94ae7b373df89e15cb25e => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:56.337461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50cba1-bcd5-4a90-b3e1-cf5e696cdbaf None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:56.338246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50cba1-bcd5-4a90-b3e1-cf5e696cdbaf None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:56.338246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50cba1-bcd5-4a90-b3e1-cf5e696cdbaf None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:56.339505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 545/1052] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:56.345230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfad864-5c99-4782-baf6-8e1159be1dbf None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:56.345714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfad864-5c99-4782-baf6-8e1159be1dbf None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:56.345793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfad864-5c99-4782-baf6-8e1159be1dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:56.365608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-4bfad864-5c99-4782-baf6-8e1159be1dbf None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:56.366835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4bfad864-5c99-4782-baf6-8e1159be1dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:56.438818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 508/1053] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:56.707806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50cba1-bcd5-4a90-b3e1-cf5e696cdbaf None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:56.745273 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:56.834918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50cba1-bcd5-4a90-b3e1-cf5e696cdbaf None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:56.878292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c8b6a-10f8-4e38-be2a-7ef83a94e427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:56.879238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c8b6a-10f8-4e38-be2a-7ef83a94e427 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:56.879696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c8b6a-10f8-4e38-be2a-7ef83a94e427 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:56.880164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c8b6a-10f8-4e38-be2a-7ef83a94e427 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:56.897039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c8b6a-10f8-4e38-be2a-7ef83a94e427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-180c8b6a-10f8-4e38-be2a-7ef83a94e427 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:56.903413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:56.904503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:41:56.905506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:56.907946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 546/1054] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:41:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:56.914081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfad864-5c99-4782-baf6-8e1159be1dbf None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:56.969425 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdadc5-c60d-4fb4-96f1-55d061b348b2 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:56.969616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdadc5-c60d-4fb4-96f1-55d061b348b2 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:56.969888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdadc5-c60d-4fb4-96f1-55d061b348b2 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:56.971352 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 547/1055] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:41:56.976377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77355891-4b17-4f6e-a075-7e9529b468ce None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:56.976656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77355891-4b17-4f6e-a075-7e9529b468ce None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:56.976929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77355891-4b17-4f6e-a075-7e9529b468ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:56.996259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-77355891-4b17-4f6e-a075-7e9529b468ce None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:56.997460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-77355891-4b17-4f6e-a075-7e9529b468ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:57.039179 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:57.071067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 548/1056] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:41:57.085508 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfad864-5c99-4782-baf6-8e1159be1dbf None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:57.122139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd566d-e792-4303-928a-d89ce3888656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:57.122745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd566d-e792-4303-928a-d89ce3888656 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:57.123027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd566d-e792-4303-928a-d89ce3888656 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:57.123299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd566d-e792-4303-928a-d89ce3888656 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:57.123976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd566d-e792-4303-928a-d89ce3888656 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:57.124364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd566d-e792-4303-928a-d89ce3888656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:57.125741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd566d-e792-4303-928a-d89ce3888656 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:57.139891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 509/1057] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:41:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:57.141295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77355891-4b17-4f6e-a075-7e9529b468ce None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:57.167824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:57.249245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77355891-4b17-4f6e-a075-7e9529b468ce None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:57.286498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da572b9-1965-4e11-ba7e-b43f5a3e24fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:57.287139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da572b9-1965-4e11-ba7e-b43f5a3e24fb None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:57.287427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da572b9-1965-4e11-ba7e-b43f5a3e24fb None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:57.287748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da572b9-1965-4e11-ba7e-b43f5a3e24fb None admin] PATH_INFO: `/v3/users/8fd40e72100f4d2d8c79b864bb9707cd` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:57.302881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da572b9-1965-4e11-ba7e-b43f5a3e24fb None admin] RBAC: Authorizing `identity:get_user(user_id=8fd40e72100f4d2d8c79b864bb9707cd)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:57.304457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da572b9-1965-4e11-ba7e-b43f5a3e24fb None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:57.311507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 549/1058] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:41:57 2022] GET /identity/v3/users/8fd40e72100f4d2d8c79b864bb9707cd => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:57.312735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd566d-e792-4303-928a-d89ce3888656 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:57.339124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da572b9-1965-4e11-ba7e-b43f5a3e24fb None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:57.430487 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd566d-e792-4303-928a-d89ce3888656 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:57.436321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:57.460148 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da572b9-1965-4e11-ba7e-b43f5a3e24fb None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:57.468702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f39b1-3825-4aa2-b2f6-8f5820432e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:57.469322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f39b1-3825-4aa2-b2f6-8f5820432e2c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:57.469597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f39b1-3825-4aa2-b2f6-8f5820432e2c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:57.469871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f39b1-3825-4aa2-b2f6-8f5820432e2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:57.472716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:57.500340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88567fa-efd1-4a54-909b-83b834c500b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:57.500833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88567fa-efd1-4a54-909b-83b834c500b3 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:57.501123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88567fa-efd1-4a54-909b-83b834c500b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:57.501392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88567fa-efd1-4a54-909b-83b834c500b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:57.543799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f39b1-3825-4aa2-b2f6-8f5820432e2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:57.545291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f39b1-3825-4aa2-b2f6-8f5820432e2c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:57.576588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88567fa-efd1-4a54-909b-83b834c500b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:57.578070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88567fa-efd1-4a54-909b-83b834c500b3 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:57.654803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 510/1059] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:41:57.654803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ce34a-cc1d-4003-a8de-2962249cd4d8 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:57.654803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ce34a-cc1d-4003-a8de-2962249cd4d8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:57.654803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ce34a-cc1d-4003-a8de-2962249cd4d8 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:57.654803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 511/1060] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:41:57.655850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ce34a-cc1d-4003-a8de-2962249cd4d8 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:57.660757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:57.681311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 550/1061] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:41:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 11:41:57.682754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef36f-7432-47e2-ac24-1c16bb9c65a3 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:57.683047 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef36f-7432-47e2-ac24-1c16bb9c65a3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:57.683332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef36f-7432-47e2-ac24-1c16bb9c65a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:57.697711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:57.707619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-577ef36f-7432-47e2-ac24-1c16bb9c65a3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:41:57.708879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-577ef36f-7432-47e2-ac24-1c16bb9c65a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:41:57.772433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ce34a-cc1d-4003-a8de-2962249cd4d8 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:57.785670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 551/1062] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:41:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:41:57.786870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ef36f-7432-47e2-ac24-1c16bb9c65a3 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:57.826466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a8b57-f64e-4193-b13c-9c9c44b71e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:57.827105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a8b57-f64e-4193-b13c-9c9c44b71e7f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:57.827390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a8b57-f64e-4193-b13c-9c9c44b71e7f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:57.827674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a8b57-f64e-4193-b13c-9c9c44b71e7f None admin] PATH_INFO: `/v3/projects/520539582f2c49db9a8962f8056f5099` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:57.828962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a8b57-f64e-4193-b13c-9c9c44b71e7f None admin] RBAC: Authorizing `identity:get_project(project_id=520539582f2c49db9a8962f8056f5099)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:57.831016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a8b57-f64e-4193-b13c-9c9c44b71e7f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:57.838609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 512/1063] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:41:57 2022] GET /identity/v3/projects/520539582f2c49db9a8962f8056f5099 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:57.840383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a8b57-f64e-4193-b13c-9c9c44b71e7f None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:57.903744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ef36f-7432-47e2-ac24-1c16bb9c65a3 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:57.919489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:57.946345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec602fb-621c-485a-a006-eb282ef857f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:57.947006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec602fb-621c-485a-a006-eb282ef857f7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:57.947264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec602fb-621c-485a-a006-eb282ef857f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:57.947595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec602fb-621c-485a-a006-eb282ef857f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:57.959043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec602fb-621c-485a-a006-eb282ef857f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:57.960601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec602fb-621c-485a-a006-eb282ef857f7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-0ec602fb-621c-485a-a006-eb282ef857f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:57.979248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:57.980345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:57.981268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:57.981268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:57.981268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:41:57.986055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 552/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:57.986806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec602fb-621c-485a-a006-eb282ef857f7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:57.994591 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a8b57-f64e-4193-b13c-9c9c44b71e7f None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:57.997156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:58.044659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894176a-b831-4e31-b858-6a182cb5961a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:58.045179 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894176a-b831-4e31-b858-6a182cb5961a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:58.045464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894176a-b831-4e31-b858-6a182cb5961a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:58.045734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894176a-b831-4e31-b858-6a182cb5961a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:58.046464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894176a-b831-4e31-b858-6a182cb5961a None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=520539582f2c49db9a8962f8056f5099, user.id=8fd40e72100f4d2d8c79b864bb9707cd) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:58.046771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894176a-b831-4e31-b858-6a182cb5961a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:58.048575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894176a-b831-4e31-b858-6a182cb5961a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:58.055325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 513/1065] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:41:57 2022] GET /identity/v3/role_assignments?user.id=8fd40e72100f4d2d8c79b864bb9707cd&scope.project.id=520539582f2c49db9a8962f8056f5099&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:58.066498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894176a-b831-4e31-b858-6a182cb5961a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:58.120713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec602fb-621c-485a-a006-eb282ef857f7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:58.165504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b292f-c8d9-44a2-a4a7-7f5f88c684eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:58.166132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b292f-c8d9-44a2-a4a7-7f5f88c684eb None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:58.166415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b292f-c8d9-44a2-a4a7-7f5f88c684eb None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:58.166690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b292f-c8d9-44a2-a4a7-7f5f88c684eb None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:58.167347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b292f-c8d9-44a2-a4a7-7f5f88c684eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:58.167655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b292f-c8d9-44a2-a4a7-7f5f88c684eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:58.169199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b292f-c8d9-44a2-a4a7-7f5f88c684eb None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:58.184113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 553/1066] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:58.209616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894176a-b831-4e31-b858-6a182cb5961a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:58.274675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9605345-f742-4fe8-962b-d91ce81cb7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:58.275331 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9605345-f742-4fe8-962b-d91ce81cb7b3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:58.275671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9605345-f742-4fe8-962b-d91ce81cb7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:58.275815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9605345-f742-4fe8-962b-d91ce81cb7b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:58.288494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9605345-f742-4fe8-962b-d91ce81cb7b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-b9605345-f742-4fe8-962b-d91ce81cb7b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:41:58.293552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:58.294608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:58.297315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 514/1067] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 11:41:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:58.301216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b292f-c8d9-44a2-a4a7-7f5f88c684eb None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:58.329842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:58.430892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b292f-c8d9-44a2-a4a7-7f5f88c684eb None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:58.471653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d9547-32df-4ffd-a613-e8de4a153732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:58.472318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9547-32df-4ffd-a613-e8de4a153732 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:58.472604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9547-32df-4ffd-a613-e8de4a153732 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:58.472879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9547-32df-4ffd-a613-e8de4a153732 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:58.473596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d9547-32df-4ffd-a613-e8de4a153732 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:58.473898 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d9547-32df-4ffd-a613-e8de4a153732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:58.475277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d9547-32df-4ffd-a613-e8de4a153732 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:58.485701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 554/1068] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 11:41:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:58.492431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9605345-f742-4fe8-962b-d91ce81cb7b3 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:58.539513 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:58.621372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9605345-f742-4fe8-962b-d91ce81cb7b3 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:58.658638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef5f88-e592-490a-ab45-b3252d176554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:58.659276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef5f88-e592-490a-ab45-b3252d176554 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:58.659554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef5f88-e592-490a-ab45-b3252d176554 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:58.659832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef5f88-e592-490a-ab45-b3252d176554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:58.671380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef5f88-e592-490a-ab45-b3252d176554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:58.672887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef5f88-e592-490a-ab45-b3252d176554 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-f3ef5f88-e592-490a-ab45-b3252d176554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:58.688195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:58.690014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:58.691794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:58.691794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:58.691794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:58.691794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 515/1069] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:58.696499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d9547-32df-4ffd-a613-e8de4a153732 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:58.797201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:58.824584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d9547-32df-4ffd-a613-e8de4a153732 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:58.862140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ad0b-b91b-4e7b-a8bb-b011aeff870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:58.862753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ad0b-b91b-4e7b-a8bb-b011aeff870c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:58.863042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ad0b-b91b-4e7b-a8bb-b011aeff870c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:58.863317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ad0b-b91b-4e7b-a8bb-b011aeff870c None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:58.863959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ad0b-b91b-4e7b-a8bb-b011aeff870c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:58.864299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ad0b-b91b-4e7b-a8bb-b011aeff870c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:58.865673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ad0b-b91b-4e7b-a8bb-b011aeff870c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:58.884226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 555/1070] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:58.885326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef5f88-e592-490a-ab45-b3252d176554 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:58.976430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:58.999085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef5f88-e592-490a-ab45-b3252d176554 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:59.040854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74674d8d-15fa-4ad2-8c37-ba1e9faddbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:59.041502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74674d8d-15fa-4ad2-8c37-ba1e9faddbd1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:59.041785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74674d8d-15fa-4ad2-8c37-ba1e9faddbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:59.042057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74674d8d-15fa-4ad2-8c37-ba1e9faddbd1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:59.050481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74674d8d-15fa-4ad2-8c37-ba1e9faddbd1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:59.051933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74674d8d-15fa-4ad2-8c37-ba1e9faddbd1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-74674d8d-15fa-4ad2-8c37-ba1e9faddbd1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:41:59.063888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:59.065085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:59.066149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 516/1071] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 11:41:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:41:59.071716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ad0b-b91b-4e7b-a8bb-b011aeff870c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:59.186871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ad0b-b91b-4e7b-a8bb-b011aeff870c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:59.228221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b15884-1825-47b1-b315-28f2c948448b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:59.228856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b15884-1825-47b1-b315-28f2c948448b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:59.229145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b15884-1825-47b1-b315-28f2c948448b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:59.229397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b15884-1825-47b1-b315-28f2c948448b None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:59.230157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b15884-1825-47b1-b315-28f2c948448b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:59.230456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b15884-1825-47b1-b315-28f2c948448b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:59.231960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b15884-1825-47b1-b315-28f2c948448b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:59.253932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 556/1072] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 11:41:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:59.260715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74674d8d-15fa-4ad2-8c37-ba1e9faddbd1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:59.386276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74674d8d-15fa-4ad2-8c37-ba1e9faddbd1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:59.421779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1b255-688f-4430-9370-4de53f580d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:59.422471 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1b255-688f-4430-9370-4de53f580d39 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:59.422748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1b255-688f-4430-9370-4de53f580d39 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:59.423022 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1b255-688f-4430-9370-4de53f580d39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:59.431165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:59.435475 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1b255-688f-4430-9370-4de53f580d39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:59.437068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1b255-688f-4430-9370-4de53f580d39 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-8bf1b255-688f-4430-9370-4de53f580d39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:41:59.453147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:59.454210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:41:59.455168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:41:59.455168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:41:59.455168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:59.455168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 517/1073] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:59.460329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b15884-1825-47b1-b315-28f2c948448b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:59.463274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:59.577857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b15884-1825-47b1-b315-28f2c948448b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:59.620039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75173c4-c59f-4f2f-bad0-fa0a0f611fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:59.620698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75173c4-c59f-4f2f-bad0-fa0a0f611fb5 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:59.620980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75173c4-c59f-4f2f-bad0-fa0a0f611fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:59.621256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75173c4-c59f-4f2f-bad0-fa0a0f611fb5 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:59.621921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75173c4-c59f-4f2f-bad0-fa0a0f611fb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:59.622228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75173c4-c59f-4f2f-bad0-fa0a0f611fb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:59.623643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75173c4-c59f-4f2f-bad0-fa0a0f611fb5 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:59.636565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 557/1074] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:41:59.642495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1b255-688f-4430-9370-4de53f580d39 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:59.747561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1b255-688f-4430-9370-4de53f580d39 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:59.781393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec6633-b7cb-41ac-b58e-cc3edfb8e4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:59.782025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec6633-b7cb-41ac-b58e-cc3edfb8e4cc None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:59.782302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec6633-b7cb-41ac-b58e-cc3edfb8e4cc None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:59.782577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec6633-b7cb-41ac-b58e-cc3edfb8e4cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:59.790314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec6633-b7cb-41ac-b58e-cc3edfb8e4cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:59.791725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec6633-b7cb-41ac-b58e-cc3edfb8e4cc None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-14ec6633-b7cb-41ac-b58e-cc3edfb8e4cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:41:59.804389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:41:59.805372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:41:59.806319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:41:59.806319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:41:59.806319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:41:59.806319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:41:59.806319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:41:59.806319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:41:59.806319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:41:59.806319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:41:59.806319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 518/1075] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:41:59.811431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75173c4-c59f-4f2f-bad0-fa0a0f611fb5 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:41:59.878924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:41:59.921058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75173c4-c59f-4f2f-bad0-fa0a0f611fb5 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:41:59.967949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c217f3-4633-4b8a-9416-17e9af784d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:41:59.968588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c217f3-4633-4b8a-9416-17e9af784d7e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:41:59.968866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c217f3-4633-4b8a-9416-17e9af784d7e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:41:59.969143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c217f3-4633-4b8a-9416-17e9af784d7e None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:41:59.969830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c217f3-4633-4b8a-9416-17e9af784d7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:41:59.970124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c217f3-4633-4b8a-9416-17e9af784d7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:41:59.971570 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c217f3-4633-4b8a-9416-17e9af784d7e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:41:59.985218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 558/1076] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:41:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:41:59.991455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec6633-b7cb-41ac-b58e-cc3edfb8e4cc None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:00.046076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:00.113309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec6633-b7cb-41ac-b58e-cc3edfb8e4cc None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:00.150881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32056d1-8ab7-4964-ab93-632f0dab6d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:00.151549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32056d1-8ab7-4964-ab93-632f0dab6d9a None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:00.151826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32056d1-8ab7-4964-ab93-632f0dab6d9a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:00.152097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32056d1-8ab7-4964-ab93-632f0dab6d9a None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/ed14c1bc105d4c5cade5813edc1f0dac/roles/87a27c6c32d94ae7b373df89e15cb25e` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:00.154319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32056d1-8ab7-4964-ab93-632f0dab6d9a None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=ed14c1bc105d4c5cade5813edc1f0dac, role_id=87a27c6c32d94ae7b373df89e15cb25e)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:00.155820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32056d1-8ab7-4964-ab93-632f0dab6d9a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:00.281181 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 519/1077] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:41:59 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/ed14c1bc105d4c5cade5813edc1f0dac/roles/87a27c6c32d94ae7b373df89e15cb25e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:42:00.297591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592529cd-ab2a-4a13-bbb8-3ef7e9045397 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:00.298072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592529cd-ab2a-4a13-bbb8-3ef7e9045397 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:00.298692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592529cd-ab2a-4a13-bbb8-3ef7e9045397 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:00.301549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 559/1078] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:00.308040 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323dd27-3ed2-474b-88ee-def539336066 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:00.308353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323dd27-3ed2-474b-88ee-def539336066 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:00.308630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323dd27-3ed2-474b-88ee-def539336066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:00.329879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b323dd27-3ed2-474b-88ee-def539336066 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:00.331119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b323dd27-3ed2-474b-88ee-def539336066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:00.416864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 520/1079] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:00.703027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592529cd-ab2a-4a13-bbb8-3ef7e9045397 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:00.852369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592529cd-ab2a-4a13-bbb8-3ef7e9045397 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:00.880421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:00.894684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bdaa5-77dd-44e2-bbdd-2638c8aae615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:00.895360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bdaa5-77dd-44e2-bbdd-2638c8aae615 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:00.895689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bdaa5-77dd-44e2-bbdd-2638c8aae615 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:00.895986 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bdaa5-77dd-44e2-bbdd-2638c8aae615 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:00.909312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bdaa5-77dd-44e2-bbdd-2638c8aae615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-0f4bdaa5-77dd-44e2-bbdd-2638c8aae615 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:42:00.915215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:00.916537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:42:00.917785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 560/1080] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:42:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:00.922872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323dd27-3ed2-474b-88ee-def539336066 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:00.965816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:01.026828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323dd27-3ed2-474b-88ee-def539336066 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:01.062415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bbcc7-5f53-4792-95e9-a03e58ac841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:01.063017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bbcc7-5f53-4792-95e9-a03e58ac841d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:01.063296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bbcc7-5f53-4792-95e9-a03e58ac841d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:01.063566 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bbcc7-5f53-4792-95e9-a03e58ac841d None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:01.064249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bbcc7-5f53-4792-95e9-a03e58ac841d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:01.064551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bbcc7-5f53-4792-95e9-a03e58ac841d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:01.065918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bbcc7-5f53-4792-95e9-a03e58ac841d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:01.075092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 521/1081] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:42:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:01.081423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bdaa5-77dd-44e2-bbdd-2638c8aae615 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:01.215410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bdaa5-77dd-44e2-bbdd-2638c8aae615 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:01.256325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ebb0b-5a92-494e-897a-7fdd3adec0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:01.257513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ebb0b-5a92-494e-897a-7fdd3adec0b7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:01.257843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ebb0b-5a92-494e-897a-7fdd3adec0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:01.258194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ebb0b-5a92-494e-897a-7fdd3adec0b7 None admin] PATH_INFO: `/v3/users/8fd40e72100f4d2d8c79b864bb9707cd` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:01.259612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ebb0b-5a92-494e-897a-7fdd3adec0b7 None admin] RBAC: Authorizing `identity:get_user(user_id=8fd40e72100f4d2d8c79b864bb9707cd)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:01.261410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ebb0b-5a92-494e-897a-7fdd3adec0b7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:01.263900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 561/1082] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:01 2022] GET /identity/v3/users/8fd40e72100f4d2d8c79b864bb9707cd => generated 285 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:01.270001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bbcc7-5f53-4792-95e9-a03e58ac841d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:01.385910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bbcc7-5f53-4792-95e9-a03e58ac841d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:01.388457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:01.427141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b8adb-193e-4491-9a47-af57810de639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:01.427779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b8adb-193e-4491-9a47-af57810de639 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:01.428060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b8adb-193e-4491-9a47-af57810de639 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:01.428494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b8adb-193e-4491-9a47-af57810de639 None admin] PATH_INFO: `/v3/projects/520539582f2c49db9a8962f8056f5099` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:01.430154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b8adb-193e-4491-9a47-af57810de639 None admin] RBAC: Authorizing `identity:get_project(project_id=520539582f2c49db9a8962f8056f5099)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:01.431611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b8adb-193e-4491-9a47-af57810de639 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:01.433464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 522/1083] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:01 2022] GET /identity/v3/projects/520539582f2c49db9a8962f8056f5099 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:01.439823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ebb0b-5a92-494e-897a-7fdd3adec0b7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:01.499618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:01.554287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ebb0b-5a92-494e-897a-7fdd3adec0b7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:01.590619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941e159-bbcd-47e8-96ca-f4169832fb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:01.591277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941e159-bbcd-47e8-96ca-f4169832fb94 None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:01.591558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941e159-bbcd-47e8-96ca-f4169832fb94 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:01.591835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941e159-bbcd-47e8-96ca-f4169832fb94 None admin] PATH_INFO: `/v3/projects/520539582f2c49db9a8962f8056f5099/users/8fd40e72100f4d2d8c79b864bb9707cd/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:01.593775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941e159-bbcd-47e8-96ca-f4169832fb94 None admin] RBAC: Authorizing `identity:create_grant(project_id=520539582f2c49db9a8962f8056f5099, user_id=8fd40e72100f4d2d8c79b864bb9707cd, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:01.595209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941e159-bbcd-47e8-96ca-f4169832fb94 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:01.709303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 562/1084] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:42:01 2022] PUT /identity/v3/projects/520539582f2c49db9a8962f8056f5099/users/8fd40e72100f4d2d8c79b864bb9707cd/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:42:02.472125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb6c33-6f3d-4db5-b9dd-14c27026887c None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:02.472690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb6c33-6f3d-4db5-b9dd-14c27026887c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:02.472690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb6c33-6f3d-4db5-b9dd-14c27026887c None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:02.474158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 523/1085] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:02.480264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32424-57ae-4c19-b779-b3a96845e5a4 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:02.480559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32424-57ae-4c19-b779-b3a96845e5a4 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:02.480832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32424-57ae-4c19-b779-b3a96845e5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:02.499985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-65b32424-57ae-4c19-b779-b3a96845e5a4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:02.501213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-65b32424-57ae-4c19-b779-b3a96845e5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:02.579760 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 563/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:02.852914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb6c33-6f3d-4db5-b9dd-14c27026887c None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:02.985459 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb6c33-6f3d-4db5-b9dd-14c27026887c None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:03.037146 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb13a9-9996-43e3-81ae-1dbfbecacf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:03.037908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb13a9-9996-43e3-81ae-1dbfbecacf1b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:03.038186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb13a9-9996-43e3-81ae-1dbfbecacf1b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:03.038462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb13a9-9996-43e3-81ae-1dbfbecacf1b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:03.050151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb13a9-9996-43e3-81ae-1dbfbecacf1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-11eb13a9-9996-43e3-81ae-1dbfbecacf1b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:42:03.055435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:03.057504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:42:03.059314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 524/1087] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Oct 4 11:42:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:42:03.064229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b32424-57ae-4c19-b779-b3a96845e5a4 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:03.117266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:03.186270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b32424-57ae-4c19-b779-b3a96845e5a4 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:03.223635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510139e-a532-4369-940e-35f5cab8a152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:03.224280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510139e-a532-4369-940e-35f5cab8a152 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:03.224564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510139e-a532-4369-940e-35f5cab8a152 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:03.224840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510139e-a532-4369-940e-35f5cab8a152 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:03.226012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510139e-a532-4369-940e-35f5cab8a152 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:03.226319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510139e-a532-4369-940e-35f5cab8a152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:03.227697 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510139e-a532-4369-940e-35f5cab8a152 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:03.236548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 564/1088] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 11:42:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:03.243084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb13a9-9996-43e3-81ae-1dbfbecacf1b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:03.283569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:03.370018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb13a9-9996-43e3-81ae-1dbfbecacf1b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:03.408058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d6581-e834-4bf2-8a2e-a3faccd3a055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:03.408745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d6581-e834-4bf2-8a2e-a3faccd3a055 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:03.409027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d6581-e834-4bf2-8a2e-a3faccd3a055 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:03.409306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d6581-e834-4bf2-8a2e-a3faccd3a055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:03.422561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d6581-e834-4bf2-8a2e-a3faccd3a055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:03.424014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d6581-e834-4bf2-8a2e-a3faccd3a055 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-b78d6581-e834-4bf2-8a2e-a3faccd3a055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:42:03.439829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:03.441807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:42:03.443815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:42:03.443815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:42:03.443815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:42:03.443815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 525/1089] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:03.449684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510139e-a532-4369-940e-35f5cab8a152 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:03.566925 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510139e-a532-4369-940e-35f5cab8a152 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:03.568599 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:03.615957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5707d-ba7b-4c57-bb1e-c6caca87d06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:03.616590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5707d-ba7b-4c57-bb1e-c6caca87d06c None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:03.616870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5707d-ba7b-4c57-bb1e-c6caca87d06c None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:03.617150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5707d-ba7b-4c57-bb1e-c6caca87d06c None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:03.617805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5707d-ba7b-4c57-bb1e-c6caca87d06c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:03.618108 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5707d-ba7b-4c57-bb1e-c6caca87d06c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:03.619478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5707d-ba7b-4c57-bb1e-c6caca87d06c None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:03.631977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 565/1090] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:03.637814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d6581-e834-4bf2-8a2e-a3faccd3a055 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:03.725564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:03.746321 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d6581-e834-4bf2-8a2e-a3faccd3a055 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:03.780991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795d1fa-cbb3-4e79-974c-2862359c407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:03.781626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795d1fa-cbb3-4e79-974c-2862359c407f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:03.781941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795d1fa-cbb3-4e79-974c-2862359c407f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:03.782207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795d1fa-cbb3-4e79-974c-2862359c407f None admin] PATH_INFO: `/v3/users/nova` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:03.792782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795d1fa-cbb3-4e79-974c-2862359c407f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:03.794268 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795d1fa-cbb3-4e79-974c-2862359c407f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-9795d1fa-cbb3-4e79-974c-2862359c407f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:42:03.807085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:03.808344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:42:03.809370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 526/1091] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Oct 4 11:42:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:03.814572 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5707d-ba7b-4c57-bb1e-c6caca87d06c None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:03.927979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5707d-ba7b-4c57-bb1e-c6caca87d06c None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:03.932288 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc81c4-0618-4f4f-9f34-6bc55f6daf81 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:03.932576 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc81c4-0618-4f4f-9f34-6bc55f6daf81 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:03.932856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc81c4-0618-4f4f-9f34-6bc55f6daf81 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:03.934202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 527/1092] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:42:03.939589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86349e3-9ab5-4f0e-a684-7d053a82f6fb None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:03.939879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86349e3-9ab5-4f0e-a684-7d053a82f6fb None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:03.940159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86349e3-9ab5-4f0e-a684-7d053a82f6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:03.960552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b86349e3-9ab5-4f0e-a684-7d053a82f6fb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:03.961773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b86349e3-9ab5-4f0e-a684-7d053a82f6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:03.975557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4277e1-33f6-4b97-a5cd-0977de573544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:03.976194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4277e1-33f6-4b97-a5cd-0977de573544 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:03.976480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4277e1-33f6-4b97-a5cd-0977de573544 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:03.976757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4277e1-33f6-4b97-a5cd-0977de573544 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:03.977487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4277e1-33f6-4b97-a5cd-0977de573544 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:03.977788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4277e1-33f6-4b97-a5cd-0977de573544 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:03.979224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4277e1-33f6-4b97-a5cd-0977de573544 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:03.997511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 566/1093] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Oct 4 11:42:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:04.003616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4277e1-33f6-4b97-a5cd-0977de573544 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:04.026934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:04.035947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 528/1094] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:42:04.110269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4277e1-33f6-4b97-a5cd-0977de573544 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:04.146442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245798d-f03b-4c69-870b-da09879ee861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:04.146999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245798d-f03b-4c69-870b-da09879ee861 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:04.147303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245798d-f03b-4c69-870b-da09879ee861 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:04.147575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245798d-f03b-4c69-870b-da09879ee861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:04.219671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245798d-f03b-4c69-870b-da09879ee861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:04.221068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245798d-f03b-4c69-870b-da09879ee861 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:04.242782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:04.306277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86349e3-9ab5-4f0e-a684-7d053a82f6fb None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:04.325552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 567/1095] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:42:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 11:42:04.331478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245798d-f03b-4c69-870b-da09879ee861 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:04.378816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:04.410324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86349e3-9ab5-4f0e-a684-7d053a82f6fb None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:04.437765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245798d-f03b-4c69-870b-da09879ee861 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:04.454391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99828678-30ca-422f-a89f-e43911a10da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:04.455045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99828678-30ca-422f-a89f-e43911a10da7 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:04.455324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99828678-30ca-422f-a89f-e43911a10da7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:04.455596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99828678-30ca-422f-a89f-e43911a10da7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:04.461005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:04.466814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99828678-30ca-422f-a89f-e43911a10da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-99828678-30ca-422f-a89f-e43911a10da7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:42:04.474472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:04.476407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:42:04.477468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 529/1096] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:42:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:42:04.478908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c249526-36ad-425f-95e5-a5f80afd5607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:04.479130 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c249526-36ad-425f-95e5-a5f80afd5607 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:04.479328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c249526-36ad-425f-95e5-a5f80afd5607 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:04.479601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c249526-36ad-425f-95e5-a5f80afd5607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:04.479910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99828678-30ca-422f-a89f-e43911a10da7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:04.490513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c249526-36ad-425f-95e5-a5f80afd5607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:04.492937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c249526-36ad-425f-95e5-a5f80afd5607 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-6c249526-36ad-425f-95e5-a5f80afd5607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:42:04.509308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:04.510992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:42:04.512519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:42:04.512519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:42:04.512519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:42:04.512519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 568/1097] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:42:04.516113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c249526-36ad-425f-95e5-a5f80afd5607 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:04.588820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99828678-30ca-422f-a89f-e43911a10da7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:04.621818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:04.626276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c249526-36ad-425f-95e5-a5f80afd5607 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:04.627235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c70e9-8334-4607-a453-7ea954f13f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:04.627847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c70e9-8334-4607-a453-7ea954f13f62 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:04.628135 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c70e9-8334-4607-a453-7ea954f13f62 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:04.628422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c70e9-8334-4607-a453-7ea954f13f62 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:04.629143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c70e9-8334-4607-a453-7ea954f13f62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:04.629447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c70e9-8334-4607-a453-7ea954f13f62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:04.630824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c70e9-8334-4607-a453-7ea954f13f62 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:04.639330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 530/1098] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:42:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:04.650745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c70e9-8334-4607-a453-7ea954f13f62 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:04.661749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab442b4-85b1-4c5c-9947-f4d73df82b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:04.662354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab442b4-85b1-4c5c-9947-f4d73df82b55 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:04.662648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab442b4-85b1-4c5c-9947-f4d73df82b55 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:04.663477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab442b4-85b1-4c5c-9947-f4d73df82b55 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:04.664126 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab442b4-85b1-4c5c-9947-f4d73df82b55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:04.664460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab442b4-85b1-4c5c-9947-f4d73df82b55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:04.665838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab442b4-85b1-4c5c-9947-f4d73df82b55 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:04.678060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 569/1099] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:04.684628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab442b4-85b1-4c5c-9947-f4d73df82b55 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:04.718326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:04.755456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c70e9-8334-4607-a453-7ea954f13f62 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:04.790630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac563909-2089-4b71-aef1-69b4f461950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:04.791374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac563909-2089-4b71-aef1-69b4f461950c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:04.791649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac563909-2089-4b71-aef1-69b4f461950c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:04.791923 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac563909-2089-4b71-aef1-69b4f461950c None admin] PATH_INFO: `/v3/users/8fd40e72100f4d2d8c79b864bb9707cd` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:04.793112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac563909-2089-4b71-aef1-69b4f461950c None admin] RBAC: Authorizing `identity:get_user(user_id=8fd40e72100f4d2d8c79b864bb9707cd)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:04.794546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac563909-2089-4b71-aef1-69b4f461950c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:04.796150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 531/1100] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:04 2022] GET /identity/v3/users/8fd40e72100f4d2d8c79b864bb9707cd => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:04.800467 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab442b4-85b1-4c5c-9947-f4d73df82b55 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:04.801860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac563909-2089-4b71-aef1-69b4f461950c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:04.837168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5c22-bd3d-4c98-b00f-99d9de7b2af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:04.837819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5c22-bd3d-4c98-b00f-99d9de7b2af1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:04.838102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5c22-bd3d-4c98-b00f-99d9de7b2af1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:04.838377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5c22-bd3d-4c98-b00f-99d9de7b2af1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:04.845530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd5c22-bd3d-4c98-b00f-99d9de7b2af1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:04.846988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd5c22-bd3d-4c98-b00f-99d9de7b2af1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-4ccd5c22-bd3d-4c98-b00f-99d9de7b2af1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:42:04.860658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:04.861664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:04.862557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:42:04.862557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:42:04.862557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:42:04.862557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:42:04.862557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:42:04.862557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:42:04.862557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:42:04.862557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:42:04.862557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 570/1101] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:42:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:04.867038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:04.867691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5c22-bd3d-4c98-b00f-99d9de7b2af1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:04.915940 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac563909-2089-4b71-aef1-69b4f461950c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:04.960091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:04.960544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f4750-8388-4254-80e9-5b1c061859c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:04.961154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f4750-8388-4254-80e9-5b1c061859c8 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:04.961442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f4750-8388-4254-80e9-5b1c061859c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:04.961709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f4750-8388-4254-80e9-5b1c061859c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:04.968742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5c22-bd3d-4c98-b00f-99d9de7b2af1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:05.005022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57149a5-c10e-4a8a-97f5-fa93f329cc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:05.005022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57149a5-c10e-4a8a-97f5-fa93f329cc7a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:05.005336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57149a5-c10e-4a8a-97f5-fa93f329cc7a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:05.005608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57149a5-c10e-4a8a-97f5-fa93f329cc7a None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:05.006253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57149a5-c10e-4a8a-97f5-fa93f329cc7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:05.007059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57149a5-c10e-4a8a-97f5-fa93f329cc7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:05.008522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57149a5-c10e-4a8a-97f5-fa93f329cc7a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:05.022324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 571/1102] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:05.030749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57149a5-c10e-4a8a-97f5-fa93f329cc7a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:05.039819 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f4750-8388-4254-80e9-5b1c061859c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:05.041370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f4750-8388-4254-80e9-5b1c061859c8 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:05.105028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:05.136934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57149a5-c10e-4a8a-97f5-fa93f329cc7a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:05.151281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 532/1103] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:42:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 11:42:05.152649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f4750-8388-4254-80e9-5b1c061859c8 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:05.177661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968cb7-479b-41f7-af5f-4c5e0a79a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:05.178273 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968cb7-479b-41f7-af5f-4c5e0a79a785 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:05.179071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968cb7-479b-41f7-af5f-4c5e0a79a785 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:05.179349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968cb7-479b-41f7-af5f-4c5e0a79a785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:05.180075 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa968cb7-479b-41f7-af5f-4c5e0a79a785 None admin] RBAC: Adding query filter params (role.id=87a27c6c32d94ae7b373df89e15cb25e, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=ed14c1bc105d4c5cade5813edc1f0dac) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:05.180417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa968cb7-479b-41f7-af5f-4c5e0a79a785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:05.181892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa968cb7-479b-41f7-af5f-4c5e0a79a785 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:05.188096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 572/1104] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:42:05 2022] GET /identity/v3/role_assignments?user.id=ed14c1bc105d4c5cade5813edc1f0dac&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=87a27c6c32d94ae7b373df89e15cb25e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:05.269041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f4750-8388-4254-80e9-5b1c061859c8 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:05.310511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a2ef5-2dde-45ad-a38f-acb405a0d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:05.311169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a2ef5-2dde-45ad-a38f-acb405a0d5df None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:05.311447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a2ef5-2dde-45ad-a38f-acb405a0d5df None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:05.311718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a2ef5-2dde-45ad-a38f-acb405a0d5df None admin] PATH_INFO: `/v3/projects/520539582f2c49db9a8962f8056f5099` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:05.313112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a2ef5-2dde-45ad-a38f-acb405a0d5df None admin] RBAC: Authorizing `identity:get_project(project_id=520539582f2c49db9a8962f8056f5099)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:05.314642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a2ef5-2dde-45ad-a38f-acb405a0d5df None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:05.316527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 533/1105] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:05 2022] GET /identity/v3/projects/520539582f2c49db9a8962f8056f5099 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:05.323749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968cb7-479b-41f7-af5f-4c5e0a79a785 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:05.345722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:05.436346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968cb7-479b-41f7-af5f-4c5e0a79a785 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:05.471961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9007c-11f0-475c-a080-69ee860efb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:05.472616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9007c-11f0-475c-a080-69ee860efb92 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:05.472898 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9007c-11f0-475c-a080-69ee860efb92 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:05.473232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9007c-11f0-475c-a080-69ee860efb92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:05.473969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9007c-11f0-475c-a080-69ee860efb92 None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=520539582f2c49db9a8962f8056f5099, user.id=8fd40e72100f4d2d8c79b864bb9707cd) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:05.474279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9007c-11f0-475c-a080-69ee860efb92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:05.476282 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9007c-11f0-475c-a080-69ee860efb92 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:05.483003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 573/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:42:05 2022] GET /identity/v3/role_assignments?user.id=8fd40e72100f4d2d8c79b864bb9707cd&scope.project.id=520539582f2c49db9a8962f8056f5099&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:07.759874 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293e3e5-9945-4aad-9611-368d4087efdf None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:07.760485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293e3e5-9945-4aad-9611-368d4087efdf None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:07.760485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293e3e5-9945-4aad-9611-368d4087efdf None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:07.761778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 534/1107] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:07.767414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc588a-988e-4190-9f8d-ce4e56e1e52a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:07.767892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc588a-988e-4190-9f8d-ce4e56e1e52a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:07.767994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc588a-988e-4190-9f8d-ce4e56e1e52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:07.790404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-05fc588a-988e-4190-9f8d-ce4e56e1e52a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:07.791938 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-05fc588a-988e-4190-9f8d-ce4e56e1e52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:07.897810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 574/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:08.179845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293e3e5-9945-4aad-9611-368d4087efdf None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:08.212360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:08.289360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293e3e5-9945-4aad-9611-368d4087efdf None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:08.325255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c5476-3391-48d8-ac25-485bd51cb4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:08.325884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c5476-3391-48d8-ac25-485bd51cb4e0 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:08.326165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c5476-3391-48d8-ac25-485bd51cb4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:08.326440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c5476-3391-48d8-ac25-485bd51cb4e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:08.327571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c5476-3391-48d8-ac25-485bd51cb4e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:08.329045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c5476-3391-48d8-ac25-485bd51cb4e0 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:08.330613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 535/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:42:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:08.337334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc588a-988e-4190-9f8d-ce4e56e1e52a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:08.353235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:08.445113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc588a-988e-4190-9f8d-ce4e56e1e52a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:08.481902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3942e3d-9c55-47b5-abc4-4d5d1a16fc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:08.482512 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3942e3d-9c55-47b5-abc4-4d5d1a16fc36 None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:08.483048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3942e3d-9c55-47b5-abc4-4d5d1a16fc36 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:08.483321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3942e3d-9c55-47b5-abc4-4d5d1a16fc36 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:08.483990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3942e3d-9c55-47b5-abc4-4d5d1a16fc36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:08.485452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3942e3d-9c55-47b5-abc4-4d5d1a16fc36 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:08.566299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 575/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:42:08 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:42:10.800999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0ce45-bd30-453d-bb08-35a85f5480b6 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:10.803845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0ce45-bd30-453d-bb08-35a85f5480b6 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:10.803845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0ce45-bd30-453d-bb08-35a85f5480b6 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:10.803845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 536/1111] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:10.808950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828cac9-7a13-43ba-a68f-a17dddde548d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:10.809408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828cac9-7a13-43ba-a68f-a17dddde548d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:10.809501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828cac9-7a13-43ba-a68f-a17dddde548d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:10.828944 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f828cac9-7a13-43ba-a68f-a17dddde548d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:10.830137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f828cac9-7a13-43ba-a68f-a17dddde548d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:10.894501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:10.902185 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 576/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:11.172434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0ce45-bd30-453d-bb08-35a85f5480b6 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:11.273692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:11.278500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0ce45-bd30-453d-bb08-35a85f5480b6 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:11.313530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c95cc5-10c9-481f-a31b-647b1ed73016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:11.314160 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c95cc5-10c9-481f-a31b-647b1ed73016 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:11.314442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c95cc5-10c9-481f-a31b-647b1ed73016 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:11.314721 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c95cc5-10c9-481f-a31b-647b1ed73016 None admin] PATH_INFO: `/v3/roles/4bb04c67143448539c5af8d8bdd621f9` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:11.316165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c95cc5-10c9-481f-a31b-647b1ed73016 None admin] RBAC: Authorizing `identity:get_role(role_id=4bb04c67143448539c5af8d8bdd621f9)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:11.318249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c95cc5-10c9-481f-a31b-647b1ed73016 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:11.318911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 537/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:11 2022] GET /identity/v3/roles/4bb04c67143448539c5af8d8bdd621f9 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:11.324933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828cac9-7a13-43ba-a68f-a17dddde548d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:11.465025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828cac9-7a13-43ba-a68f-a17dddde548d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:11.511094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e966a72-83f1-4c99-8626-7b9ed2766127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:11.511822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e966a72-83f1-4c99-8626-7b9ed2766127 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:11.512115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e966a72-83f1-4c99-8626-7b9ed2766127 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:11.512509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e966a72-83f1-4c99-8626-7b9ed2766127 None admin] PATH_INFO: `/v3/users/f05aec1f796d4b579a3cbffc90966f6a` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:11.530315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e966a72-83f1-4c99-8626-7b9ed2766127 None admin] RBAC: Authorizing `identity:get_user(user_id=f05aec1f796d4b579a3cbffc90966f6a)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:11.531896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e966a72-83f1-4c99-8626-7b9ed2766127 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:11.533876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 577/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:11 2022] GET /identity/v3/users/f05aec1f796d4b579a3cbffc90966f6a => generated 286 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:11.540822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c95cc5-10c9-481f-a31b-647b1ed73016 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:11.666698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c95cc5-10c9-481f-a31b-647b1ed73016 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:11.710604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044b019-4677-4aa7-98b8-aade4514b60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:11.711260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044b019-4677-4aa7-98b8-aade4514b60e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:11.711546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044b019-4677-4aa7-98b8-aade4514b60e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:11.711821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044b019-4677-4aa7-98b8-aade4514b60e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:11.755231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:11.794583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044b019-4677-4aa7-98b8-aade4514b60e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:11.796525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044b019-4677-4aa7-98b8-aade4514b60e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:11.905256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 538/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:42:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:42:11.912368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e966a72-83f1-4c99-8626-7b9ed2766127 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:11.974009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:12.024962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e966a72-83f1-4c99-8626-7b9ed2766127 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:12.066969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8abb7e-ce6d-4b6e-8fd1-e6197253c8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:12.067638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8abb7e-ce6d-4b6e-8fd1-e6197253c8ba None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:12.067951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8abb7e-ce6d-4b6e-8fd1-e6197253c8ba None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:12.068285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8abb7e-ce6d-4b6e-8fd1-e6197253c8ba None admin] PATH_INFO: `/v3/projects/520539582f2c49db9a8962f8056f5099` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:12.069629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8abb7e-ce6d-4b6e-8fd1-e6197253c8ba None admin] RBAC: Authorizing `identity:get_project(project_id=520539582f2c49db9a8962f8056f5099)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:12.072442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8abb7e-ce6d-4b6e-8fd1-e6197253c8ba None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:12.074193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 578/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:11 2022] GET /identity/v3/projects/520539582f2c49db9a8962f8056f5099 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:12.080680 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044b019-4677-4aa7-98b8-aade4514b60e None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:12.149045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:12.183643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044b019-4677-4aa7-98b8-aade4514b60e None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:12.219973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a36b5b-511f-4fe9-8e7b-8f7bd445ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:12.220642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36b5b-511f-4fe9-8e7b-8f7bd445ccd3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:12.220967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36b5b-511f-4fe9-8e7b-8f7bd445ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:12.221284 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36b5b-511f-4fe9-8e7b-8f7bd445ccd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:12.222055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a36b5b-511f-4fe9-8e7b-8f7bd445ccd3 None admin] RBAC: Adding query filter params (role.id=4bb04c67143448539c5af8d8bdd621f9, scope.project.id=520539582f2c49db9a8962f8056f5099, user.id=f05aec1f796d4b579a3cbffc90966f6a) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:12.222405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a36b5b-511f-4fe9-8e7b-8f7bd445ccd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:12.223905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a36b5b-511f-4fe9-8e7b-8f7bd445ccd3 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:12.229912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 539/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:42:12 2022] GET /identity/v3/role_assignments?user.id=f05aec1f796d4b579a3cbffc90966f6a&scope.project.id=520539582f2c49db9a8962f8056f5099&role.id=4bb04c67143448539c5af8d8bdd621f9 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:14.380662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db541242-d0e2-448e-abe3-7bdc157b0c53 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:14.381211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db541242-d0e2-448e-abe3-7bdc157b0c53 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:14.381211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db541242-d0e2-448e-abe3-7bdc157b0c53 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:14.382526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 579/1118] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:14.388630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6e7d3-af88-4d62-a6f7-7aa47456aceb None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:14.389088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6e7d3-af88-4d62-a6f7-7aa47456aceb None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:14.389180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6e7d3-af88-4d62-a6f7-7aa47456aceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:14.408072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-fbb6e7d3-af88-4d62-a6f7-7aa47456aceb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:14.409330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-fbb6e7d3-af88-4d62-a6f7-7aa47456aceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:14.481022 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 540/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:14.751983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db541242-d0e2-448e-abe3-7bdc157b0c53 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:14.880766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db541242-d0e2-448e-abe3-7bdc157b0c53 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:14.905674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:14.917766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036964f4-ab8b-49c2-9c35-b64c585d492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:14.918394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036964f4-ab8b-49c2-9c35-b64c585d492f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:14.925113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036964f4-ab8b-49c2-9c35-b64c585d492f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:14.925113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036964f4-ab8b-49c2-9c35-b64c585d492f None admin] PATH_INFO: `/v3/roles/4bb04c67143448539c5af8d8bdd621f9` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:14.925113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036964f4-ab8b-49c2-9c35-b64c585d492f None admin] RBAC: Authorizing `identity:get_role(role_id=4bb04c67143448539c5af8d8bdd621f9)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:14.925113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036964f4-ab8b-49c2-9c35-b64c585d492f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:14.925113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 580/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:14 2022] GET /identity/v3/roles/4bb04c67143448539c5af8d8bdd621f9 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:14.929529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6e7d3-af88-4d62-a6f7-7aa47456aceb None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:14.988809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:15.032336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6e7d3-af88-4d62-a6f7-7aa47456aceb None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:15.070704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9a74-df79-4de2-a1f7-e4a10ee2117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:15.071337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9a74-df79-4de2-a1f7-e4a10ee2117a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:15.071621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9a74-df79-4de2-a1f7-e4a10ee2117a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:15.071893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9a74-df79-4de2-a1f7-e4a10ee2117a None admin] PATH_INFO: `/v3/users/f05aec1f796d4b579a3cbffc90966f6a` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:15.073075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d9a74-df79-4de2-a1f7-e4a10ee2117a None admin] RBAC: Authorizing `identity:get_user(user_id=f05aec1f796d4b579a3cbffc90966f6a)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:15.074523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d9a74-df79-4de2-a1f7-e4a10ee2117a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:15.076121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 541/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:14 2022] GET /identity/v3/users/f05aec1f796d4b579a3cbffc90966f6a => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:15.082111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036964f4-ab8b-49c2-9c35-b64c585d492f None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:15.182541 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036964f4-ab8b-49c2-9c35-b64c585d492f None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:15.217386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99745c-f55c-4aa4-bef2-11fcfd65f005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:15.218022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99745c-f55c-4aa4-bef2-11fcfd65f005 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:15.218305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99745c-f55c-4aa4-bef2-11fcfd65f005 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:15.218582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99745c-f55c-4aa4-bef2-11fcfd65f005 None admin] PATH_INFO: `/v3/projects/520539582f2c49db9a8962f8056f5099` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:15.219705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99745c-f55c-4aa4-bef2-11fcfd65f005 None admin] RBAC: Authorizing `identity:get_project(project_id=520539582f2c49db9a8962f8056f5099)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:15.221165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99745c-f55c-4aa4-bef2-11fcfd65f005 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:15.222799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 581/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:15 2022] GET /identity/v3/projects/520539582f2c49db9a8962f8056f5099 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:15.229343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9a74-df79-4de2-a1f7-e4a10ee2117a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:15.341516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9a74-df79-4de2-a1f7-e4a10ee2117a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:15.369311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:15.377323 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08486be9-21c3-4869-b4e9-609f53fbfbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:15.377948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08486be9-21c3-4869-b4e9-609f53fbfbd0 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:15.378223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08486be9-21c3-4869-b4e9-609f53fbfbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:15.385063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08486be9-21c3-4869-b4e9-609f53fbfbd0 None admin] PATH_INFO: `/v3/projects/520539582f2c49db9a8962f8056f5099/users/f05aec1f796d4b579a3cbffc90966f6a/roles/4bb04c67143448539c5af8d8bdd621f9` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:15.385063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08486be9-21c3-4869-b4e9-609f53fbfbd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=520539582f2c49db9a8962f8056f5099, user_id=f05aec1f796d4b579a3cbffc90966f6a, role_id=4bb04c67143448539c5af8d8bdd621f9)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:15.385063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08486be9-21c3-4869-b4e9-609f53fbfbd0 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:15.498465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 542/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:42:15 2022] PUT /identity/v3/projects/520539582f2c49db9a8962f8056f5099/users/f05aec1f796d4b579a3cbffc90966f6a/roles/4bb04c67143448539c5af8d8bdd621f9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:42:17.687277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ac6c2-05a0-4683-94af-dcde27f7fa3c None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:17.687863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ac6c2-05a0-4683-94af-dcde27f7fa3c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:17.687863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ac6c2-05a0-4683-94af-dcde27f7fa3c None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:17.689203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 582/1124] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:17.695520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c117d7e-be14-447a-8d04-5933d92d7e92 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:17.696005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c117d7e-be14-447a-8d04-5933d92d7e92 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:17.696375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c117d7e-be14-447a-8d04-5933d92d7e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:17.715272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-1c117d7e-be14-447a-8d04-5933d92d7e92 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:17.716513 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1c117d7e-be14-447a-8d04-5933d92d7e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:17.787924 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 543/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:18.063054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ac6c2-05a0-4683-94af-dcde27f7fa3c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:18.127188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:18.172078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ac6c2-05a0-4683-94af-dcde27f7fa3c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:18.206087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9207d98-f1d3-450c-8c3e-f7a76bd2356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:18.206705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207d98-f1d3-450c-8c3e-f7a76bd2356f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:18.206988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207d98-f1d3-450c-8c3e-f7a76bd2356f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:18.207263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207d98-f1d3-450c-8c3e-f7a76bd2356f None admin] PATH_INFO: `/v3/roles/4bb04c67143448539c5af8d8bdd621f9` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:18.208674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9207d98-f1d3-450c-8c3e-f7a76bd2356f None admin] RBAC: Authorizing `identity:get_role(role_id=4bb04c67143448539c5af8d8bdd621f9)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:18.210645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9207d98-f1d3-450c-8c3e-f7a76bd2356f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:18.211982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 583/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:18 2022] GET /identity/v3/roles/4bb04c67143448539c5af8d8bdd621f9 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:18.218373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c117d7e-be14-447a-8d04-5933d92d7e92 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:18.316713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c117d7e-be14-447a-8d04-5933d92d7e92 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:18.336064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:18.353199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fae055-365e-4563-a50d-a998ca385dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:18.354640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fae055-365e-4563-a50d-a998ca385dbc None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:18.354640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fae055-365e-4563-a50d-a998ca385dbc None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:18.354640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fae055-365e-4563-a50d-a998ca385dbc None admin] PATH_INFO: `/v3/users/f05aec1f796d4b579a3cbffc90966f6a` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:18.355618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fae055-365e-4563-a50d-a998ca385dbc None admin] RBAC: Authorizing `identity:get_user(user_id=f05aec1f796d4b579a3cbffc90966f6a)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:18.357099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fae055-365e-4563-a50d-a998ca385dbc None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:18.358990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 544/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:18 2022] GET /identity/v3/users/f05aec1f796d4b579a3cbffc90966f6a => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:18.365265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9207d98-f1d3-450c-8c3e-f7a76bd2356f None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:18.472187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9207d98-f1d3-450c-8c3e-f7a76bd2356f None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:18.498733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:18.507933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11808f65-cceb-432f-a8ec-590a9be4ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:18.509113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11808f65-cceb-432f-a8ec-590a9be4ff3d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:18.509113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11808f65-cceb-432f-a8ec-590a9be4ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:18.509327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11808f65-cceb-432f-a8ec-590a9be4ff3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:18.577343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11808f65-cceb-432f-a8ec-590a9be4ff3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:18.578700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11808f65-cceb-432f-a8ec-590a9be4ff3d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:18.676463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 584/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:42:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 11:42:18.683196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fae055-365e-4563-a50d-a998ca385dbc None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:18.786577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fae055-365e-4563-a50d-a998ca385dbc None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:18.821982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e2f22-2d69-4916-bafe-ac5acec51f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:18.822607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e2f22-2d69-4916-bafe-ac5acec51f89 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:18.822887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e2f22-2d69-4916-bafe-ac5acec51f89 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:18.823160 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e2f22-2d69-4916-bafe-ac5acec51f89 None admin] PATH_INFO: `/v3/projects/520539582f2c49db9a8962f8056f5099` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:18.824296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e2f22-2d69-4916-bafe-ac5acec51f89 None admin] RBAC: Authorizing `identity:get_project(project_id=520539582f2c49db9a8962f8056f5099)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:18.825744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e2f22-2d69-4916-bafe-ac5acec51f89 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:18.827487 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 545/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:18 2022] GET /identity/v3/projects/520539582f2c49db9a8962f8056f5099 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:18.833634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11808f65-cceb-432f-a8ec-590a9be4ff3d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:18.859923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:18.931286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11808f65-cceb-432f-a8ec-590a9be4ff3d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:18.965446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678dcdc-454f-48d6-9649-9f69c855db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:18.966050 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678dcdc-454f-48d6-9649-9f69c855db17 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:18.966333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678dcdc-454f-48d6-9649-9f69c855db17 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:18.966607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678dcdc-454f-48d6-9649-9f69c855db17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:18.967329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678dcdc-454f-48d6-9649-9f69c855db17 None admin] RBAC: Adding query filter params (role.id=4bb04c67143448539c5af8d8bdd621f9, scope.project.id=520539582f2c49db9a8962f8056f5099, user.id=f05aec1f796d4b579a3cbffc90966f6a) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:18.967638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678dcdc-454f-48d6-9649-9f69c855db17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:18.969103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678dcdc-454f-48d6-9649-9f69c855db17 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:18.975397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 585/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:42:18 2022] GET /identity/v3/role_assignments?user.id=f05aec1f796d4b579a3cbffc90966f6a&scope.project.id=520539582f2c49db9a8962f8056f5099&role.id=4bb04c67143448539c5af8d8bdd621f9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:21.186055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16018b-d4a0-4e67-977e-4406246c6dd5 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:21.186632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16018b-d4a0-4e67-977e-4406246c6dd5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:21.186632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16018b-d4a0-4e67-977e-4406246c6dd5 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:21.187958 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 546/1131] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:21.194523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573b37c-de70-4bed-9c47-fdff35cec3d1 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:21.195092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573b37c-de70-4bed-9c47-fdff35cec3d1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:21.195092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573b37c-de70-4bed-9c47-fdff35cec3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:21.214299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e573b37c-de70-4bed-9c47-fdff35cec3d1 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:21.215565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e573b37c-de70-4bed-9c47-fdff35cec3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:21.286102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 586/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:21.562989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16018b-d4a0-4e67-977e-4406246c6dd5 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:21.620642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:21.666622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16018b-d4a0-4e67-977e-4406246c6dd5 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:21.699810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6574fb-b89a-40c1-b040-bfaf7d60eb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:21.700460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6574fb-b89a-40c1-b040-bfaf7d60eb6b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:21.700743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6574fb-b89a-40c1-b040-bfaf7d60eb6b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:21.701020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6574fb-b89a-40c1-b040-bfaf7d60eb6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:21.702151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6574fb-b89a-40c1-b040-bfaf7d60eb6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:21.703609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6574fb-b89a-40c1-b040-bfaf7d60eb6b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:21.705274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 547/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:42:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:21.714507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573b37c-de70-4bed-9c47-fdff35cec3d1 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:21.736588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:21.816005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573b37c-de70-4bed-9c47-fdff35cec3d1 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:21.850794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222807b-b01b-4b92-8407-abb03956a35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:21.851409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222807b-b01b-4b92-8407-abb03956a35a None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:21.851691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222807b-b01b-4b92-8407-abb03956a35a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:21.851962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222807b-b01b-4b92-8407-abb03956a35a None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:21.852657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222807b-b01b-4b92-8407-abb03956a35a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:21.854090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222807b-b01b-4b92-8407-abb03956a35a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:21.939100 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 587/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:42:21 2022] POST /identity/v3/projects => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:42:24.162098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973a5bf-5fd6-4b07-8823-56c45759f467 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:24.162641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973a5bf-5fd6-4b07-8823-56c45759f467 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:24.162641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973a5bf-5fd6-4b07-8823-56c45759f467 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:24.163975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 548/1135] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:24.169314 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96f34e-c56f-4e6f-8794-d1cdae9aee3b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:24.169799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96f34e-c56f-4e6f-8794-d1cdae9aee3b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:24.169893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96f34e-c56f-4e6f-8794-d1cdae9aee3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:24.189117 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9d96f34e-c56f-4e6f-8794-d1cdae9aee3b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:24.190317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9d96f34e-c56f-4e6f-8794-d1cdae9aee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:24.256562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 588/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:24.523608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973a5bf-5fd6-4b07-8823-56c45759f467 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:24.622791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973a5bf-5fd6-4b07-8823-56c45759f467 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:24.645016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:24.656724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9093-c550-4688-9fec-0b1174a3d050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:24.661888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9093-c550-4688-9fec-0b1174a3d050 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:24.661888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9093-c550-4688-9fec-0b1174a3d050 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:24.661888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9093-c550-4688-9fec-0b1174a3d050 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:24.661888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a9093-c550-4688-9fec-0b1174a3d050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:24.661888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a9093-c550-4688-9fec-0b1174a3d050 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:24.662971 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 549/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:42:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:24.668698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96f34e-c56f-4e6f-8794-d1cdae9aee3b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:24.670860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:24.768585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96f34e-c56f-4e6f-8794-d1cdae9aee3b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:24.805248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5c1e6-511c-4261-9ae9-69d1ada431aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:24.805866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5c1e6-511c-4261-9ae9-69d1ada431aa None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:24.806145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5c1e6-511c-4261-9ae9-69d1ada431aa None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:24.806416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5c1e6-511c-4261-9ae9-69d1ada431aa None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:24.807079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5c1e6-511c-4261-9ae9-69d1ada431aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:24.808566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5c1e6-511c-4261-9ae9-69d1ada431aa None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:24.897909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 589/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:42:24 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:42:27.113509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44158398-4596-43f7-969b-e3e661bb8dbb None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:27.114045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44158398-4596-43f7-969b-e3e661bb8dbb None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:27.114045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44158398-4596-43f7-969b-e3e661bb8dbb None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:27.115399 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 550/1139] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:27.121034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206f1e3-ef84-47e8-93ec-91c5445a79ff None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:27.121464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206f1e3-ef84-47e8-93ec-91c5445a79ff None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:27.121588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206f1e3-ef84-47e8-93ec-91c5445a79ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:27.139658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f206f1e3-ef84-47e8-93ec-91c5445a79ff None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:27.140878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f206f1e3-ef84-47e8-93ec-91c5445a79ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:27.193017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:27.206912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 590/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:27.479873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44158398-4596-43f7-969b-e3e661bb8dbb None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:27.589982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44158398-4596-43f7-969b-e3e661bb8dbb None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:27.626576 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62887596-0d1e-4a94-9e3b-8209ef66ccfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:27.627214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887596-0d1e-4a94-9e3b-8209ef66ccfe None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:27.627490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887596-0d1e-4a94-9e3b-8209ef66ccfe None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:27.627761 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887596-0d1e-4a94-9e3b-8209ef66ccfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:27.639703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62887596-0d1e-4a94-9e3b-8209ef66ccfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-62887596-0d1e-4a94-9e3b-8209ef66ccfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:42:27.644675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:27.645734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:42:27.646648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 551/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:42:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:27.652330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206f1e3-ef84-47e8-93ec-91c5445a79ff None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:27.749391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206f1e3-ef84-47e8-93ec-91c5445a79ff None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:27.783173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119c47d-49a4-4ebb-8d88-0a35acb78784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:27.783780 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119c47d-49a4-4ebb-8d88-0a35acb78784 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:27.784057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119c47d-49a4-4ebb-8d88-0a35acb78784 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:27.784357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119c47d-49a4-4ebb-8d88-0a35acb78784 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:27.785017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119c47d-49a4-4ebb-8d88-0a35acb78784 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:27.785316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119c47d-49a4-4ebb-8d88-0a35acb78784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:27.786664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119c47d-49a4-4ebb-8d88-0a35acb78784 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:27.795809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 591/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:42:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:27.801499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62887596-0d1e-4a94-9e3b-8209ef66ccfe None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:27.852363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:27.902617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62887596-0d1e-4a94-9e3b-8209ef66ccfe None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:27.936675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ac220-1859-4031-b847-1f81b0a67a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:27.937302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ac220-1859-4031-b847-1f81b0a67a74 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:27.937584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ac220-1859-4031-b847-1f81b0a67a74 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:27.937860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ac220-1859-4031-b847-1f81b0a67a74 None admin] PATH_INFO: `/v3/users/70da35d3fa504a9b85acbf4733439894` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:27.952619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ac220-1859-4031-b847-1f81b0a67a74 None admin] RBAC: Authorizing `identity:get_user(user_id=70da35d3fa504a9b85acbf4733439894)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:27.954099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ac220-1859-4031-b847-1f81b0a67a74 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:27.955740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 552/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:27 2022] GET /identity/v3/users/70da35d3fa504a9b85acbf4733439894 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:27.961882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119c47d-49a4-4ebb-8d88-0a35acb78784 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:28.012556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:28.062022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119c47d-49a4-4ebb-8d88-0a35acb78784 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:28.094028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273593bb-1be3-46eb-8923-1d6828ad73e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:28.094637 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273593bb-1be3-46eb-8923-1d6828ad73e8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:28.094917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273593bb-1be3-46eb-8923-1d6828ad73e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:28.095188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273593bb-1be3-46eb-8923-1d6828ad73e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:28.161328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273593bb-1be3-46eb-8923-1d6828ad73e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:28.163313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273593bb-1be3-46eb-8923-1d6828ad73e8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:28.210384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:28.260637 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 592/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:42:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 11:42:28.266649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ac220-1859-4031-b847-1f81b0a67a74 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:28.368067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ac220-1859-4031-b847-1f81b0a67a74 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:28.377683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:28.402437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76813ab-3478-4090-894a-7aea455f9bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:28.403060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76813ab-3478-4090-894a-7aea455f9bc9 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:28.403339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76813ab-3478-4090-894a-7aea455f9bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:28.403610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76813ab-3478-4090-894a-7aea455f9bc9 None admin] PATH_INFO: `/v3/projects/f536cf50658943feb4d7b43f2aa81b53` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:28.404751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76813ab-3478-4090-894a-7aea455f9bc9 None admin] RBAC: Authorizing `identity:get_project(project_id=f536cf50658943feb4d7b43f2aa81b53)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:28.406181 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76813ab-3478-4090-894a-7aea455f9bc9 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:28.407805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 553/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:28 2022] GET /identity/v3/projects/f536cf50658943feb4d7b43f2aa81b53 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:28.414795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273593bb-1be3-46eb-8923-1d6828ad73e8 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:28.510861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273593bb-1be3-46eb-8923-1d6828ad73e8 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:28.543829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517986a-664a-409c-8ec9-f9c6206e01cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:28.544374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517986a-664a-409c-8ec9-f9c6206e01cc None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:28.544650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517986a-664a-409c-8ec9-f9c6206e01cc None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:28.544919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517986a-664a-409c-8ec9-f9c6206e01cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:28.545624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517986a-664a-409c-8ec9-f9c6206e01cc None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=f536cf50658943feb4d7b43f2aa81b53, user.id=70da35d3fa504a9b85acbf4733439894) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:28.545932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517986a-664a-409c-8ec9-f9c6206e01cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:28.547337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517986a-664a-409c-8ec9-f9c6206e01cc None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:28.552867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 593/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:42:28 2022] GET /identity/v3/role_assignments?user.id=70da35d3fa504a9b85acbf4733439894&scope.project.id=f536cf50658943feb4d7b43f2aa81b53&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:30.710189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025676d9-0326-4262-8614-df4819ad244e None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:30.710730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025676d9-0326-4262-8614-df4819ad244e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:30.710730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025676d9-0326-4262-8614-df4819ad244e None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:30.712065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 554/1147] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:30.717488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd3be6-c4a9-423e-9f97-c001d0848a7b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:30.717932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd3be6-c4a9-423e-9f97-c001d0848a7b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:30.718030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd3be6-c4a9-423e-9f97-c001d0848a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:30.736279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-42dd3be6-c4a9-423e-9f97-c001d0848a7b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:30.737498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-42dd3be6-c4a9-423e-9f97-c001d0848a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:30.747551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:30.806273 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 594/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:31.087800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025676d9-0326-4262-8614-df4819ad244e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:31.188710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025676d9-0326-4262-8614-df4819ad244e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:31.222147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8446e-875d-4efd-b950-79a37ff7a2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:31.222771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8446e-875d-4efd-b950-79a37ff7a2d1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:31.223051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8446e-875d-4efd-b950-79a37ff7a2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:31.223324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8446e-875d-4efd-b950-79a37ff7a2d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:31.234084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8446e-875d-4efd-b950-79a37ff7a2d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-82d8446e-875d-4efd-b950-79a37ff7a2d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:42:31.238871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:31.240114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:42:31.241397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 555/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:42:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:31.246168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd3be6-c4a9-423e-9f97-c001d0848a7b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:31.344190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd3be6-c4a9-423e-9f97-c001d0848a7b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:31.380715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2585c1-e1b3-4c56-a081-0e07d78d8031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:31.381849 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2585c1-e1b3-4c56-a081-0e07d78d8031 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:31.382124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2585c1-e1b3-4c56-a081-0e07d78d8031 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:31.382393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2585c1-e1b3-4c56-a081-0e07d78d8031 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:31.383049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2585c1-e1b3-4c56-a081-0e07d78d8031 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:31.383350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2585c1-e1b3-4c56-a081-0e07d78d8031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:31.384746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2585c1-e1b3-4c56-a081-0e07d78d8031 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:31.392845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 595/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:42:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:31.398877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8446e-875d-4efd-b950-79a37ff7a2d1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:31.433635 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:31.496757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8446e-875d-4efd-b950-79a37ff7a2d1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:31.530283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e7756-bb39-41e3-84f2-32754626404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:31.530905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e7756-bb39-41e3-84f2-32754626404e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:31.531184 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e7756-bb39-41e3-84f2-32754626404e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:31.531461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e7756-bb39-41e3-84f2-32754626404e None admin] PATH_INFO: `/v3/users/70da35d3fa504a9b85acbf4733439894` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:31.532631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e7756-bb39-41e3-84f2-32754626404e None admin] RBAC: Authorizing `identity:get_user(user_id=70da35d3fa504a9b85acbf4733439894)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:31.534098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e7756-bb39-41e3-84f2-32754626404e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:31.535768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 556/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:31 2022] GET /identity/v3/users/70da35d3fa504a9b85acbf4733439894 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:31.541989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2585c1-e1b3-4c56-a081-0e07d78d8031 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:31.551615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:31.648030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2585c1-e1b3-4c56-a081-0e07d78d8031 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:31.682783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404cf8d-36c6-488d-9794-2d6c6799465f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:31.683406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404cf8d-36c6-488d-9794-2d6c6799465f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:31.683685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404cf8d-36c6-488d-9794-2d6c6799465f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:31.683960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404cf8d-36c6-488d-9794-2d6c6799465f None admin] PATH_INFO: `/v3/projects/f536cf50658943feb4d7b43f2aa81b53` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:31.685127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404cf8d-36c6-488d-9794-2d6c6799465f None admin] RBAC: Authorizing `identity:get_project(project_id=f536cf50658943feb4d7b43f2aa81b53)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:31.686582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404cf8d-36c6-488d-9794-2d6c6799465f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:31.688258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 596/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:31 2022] GET /identity/v3/projects/f536cf50658943feb4d7b43f2aa81b53 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:31.695343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e7756-bb39-41e3-84f2-32754626404e None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:31.799427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e7756-bb39-41e3-84f2-32754626404e None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:31.801793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:31.834646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8fece-722d-4d6c-901b-6ad729f596c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:31.835833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8fece-722d-4d6c-901b-6ad729f596c2 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:31.836279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8fece-722d-4d6c-901b-6ad729f596c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:31.836570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8fece-722d-4d6c-901b-6ad729f596c2 None admin] PATH_INFO: `/v3/projects/f536cf50658943feb4d7b43f2aa81b53/users/70da35d3fa504a9b85acbf4733439894/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:31.838686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8fece-722d-4d6c-901b-6ad729f596c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f536cf50658943feb4d7b43f2aa81b53, user_id=70da35d3fa504a9b85acbf4733439894, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:31.840136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8fece-722d-4d6c-901b-6ad729f596c2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:31.932285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 557/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:42:31 2022] PUT /identity/v3/projects/f536cf50658943feb4d7b43f2aa81b53/users/70da35d3fa504a9b85acbf4733439894/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:42:34.094384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390738-db02-4a4a-871b-df22771c9cb4 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:34.094942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390738-db02-4a4a-871b-df22771c9cb4 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:34.094942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390738-db02-4a4a-871b-df22771c9cb4 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:34.096299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 597/1154] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:34.101994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83bca6-da10-4899-84e5-730a42e2c278 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:34.102431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83bca6-da10-4899-84e5-730a42e2c278 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:34.102559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83bca6-da10-4899-84e5-730a42e2c278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:34.120998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-ed83bca6-da10-4899-84e5-730a42e2c278 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:34.122180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ed83bca6-da10-4899-84e5-730a42e2c278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:34.188478 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 558/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:34.460479 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83390738-db02-4a4a-871b-df22771c9cb4 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:34.535184 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:34.560523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83390738-db02-4a4a-871b-df22771c9cb4 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:34.593095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f92f3-c476-4def-88bc-688719edd9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:34.593711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f92f3-c476-4def-88bc-688719edd9ff None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:34.593993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f92f3-c476-4def-88bc-688719edd9ff None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:34.594269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f92f3-c476-4def-88bc-688719edd9ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:34.604601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f92f3-c476-4def-88bc-688719edd9ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-e50f92f3-c476-4def-88bc-688719edd9ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:42:34.609650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:34.610810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:42:34.611746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 598/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:42:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:34.617370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83bca6-da10-4899-84e5-730a42e2c278 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:34.708914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:34.715551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83bca6-da10-4899-84e5-730a42e2c278 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:34.752540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dfe23-68d6-4432-aee4-78180af109b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:34.753135 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dfe23-68d6-4432-aee4-78180af109b8 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:34.753416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dfe23-68d6-4432-aee4-78180af109b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:34.753689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dfe23-68d6-4432-aee4-78180af109b8 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:34.754350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dfe23-68d6-4432-aee4-78180af109b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:34.754654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dfe23-68d6-4432-aee4-78180af109b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:34.756598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dfe23-68d6-4432-aee4-78180af109b8 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:34.765025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 559/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:34.771551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f92f3-c476-4def-88bc-688719edd9ff None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:34.872484 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f92f3-c476-4def-88bc-688719edd9ff None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:34.905882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05561053-24fd-4b4e-88c6-38d8504c779b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:34.906511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05561053-24fd-4b4e-88c6-38d8504c779b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:34.906795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05561053-24fd-4b4e-88c6-38d8504c779b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:34.907072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05561053-24fd-4b4e-88c6-38d8504c779b None admin] PATH_INFO: `/v3/users/70da35d3fa504a9b85acbf4733439894` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:34.908211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05561053-24fd-4b4e-88c6-38d8504c779b None admin] RBAC: Authorizing `identity:get_user(user_id=70da35d3fa504a9b85acbf4733439894)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:34.909637 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05561053-24fd-4b4e-88c6-38d8504c779b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:34.911207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 599/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:34 2022] GET /identity/v3/users/70da35d3fa504a9b85acbf4733439894 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:34.917120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dfe23-68d6-4432-aee4-78180af109b8 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:35.016423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dfe23-68d6-4432-aee4-78180af109b8 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:35.054794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6266c0-ce8e-4377-a626-0816c17f882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:35.055398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6266c0-ce8e-4377-a626-0816c17f882a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:35.055678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6266c0-ce8e-4377-a626-0816c17f882a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:35.055953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6266c0-ce8e-4377-a626-0816c17f882a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:35.080838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:35.122657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6266c0-ce8e-4377-a626-0816c17f882a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:35.124120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6266c0-ce8e-4377-a626-0816c17f882a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:35.221500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 560/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:42:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:42:35.227715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05561053-24fd-4b4e-88c6-38d8504c779b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:35.230702 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:35.338933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05561053-24fd-4b4e-88c6-38d8504c779b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:35.372544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44875bc-b40b-4cb8-8ee6-89a8ab28d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:35.373174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44875bc-b40b-4cb8-8ee6-89a8ab28d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:35.373450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44875bc-b40b-4cb8-8ee6-89a8ab28d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:35.373792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44875bc-b40b-4cb8-8ee6-89a8ab28d5c5 None admin] PATH_INFO: `/v3/projects/f536cf50658943feb4d7b43f2aa81b53` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:35.374856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44875bc-b40b-4cb8-8ee6-89a8ab28d5c5 None admin] RBAC: Authorizing `identity:get_project(project_id=f536cf50658943feb4d7b43f2aa81b53)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:35.376420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44875bc-b40b-4cb8-8ee6-89a8ab28d5c5 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:35.378087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 600/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:35 2022] GET /identity/v3/projects/f536cf50658943feb4d7b43f2aa81b53 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:35.385069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6266c0-ce8e-4377-a626-0816c17f882a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:35.443583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:35.490564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6266c0-ce8e-4377-a626-0816c17f882a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:35.526286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54ea10-3be3-45a3-a642-9cb6af54d568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:35.526919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54ea10-3be3-45a3-a642-9cb6af54d568 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:35.527199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54ea10-3be3-45a3-a642-9cb6af54d568 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:35.527476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54ea10-3be3-45a3-a642-9cb6af54d568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:35.528233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54ea10-3be3-45a3-a642-9cb6af54d568 None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=f536cf50658943feb4d7b43f2aa81b53, user.id=70da35d3fa504a9b85acbf4733439894) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:35.528548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54ea10-3be3-45a3-a642-9cb6af54d568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:35.530037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54ea10-3be3-45a3-a642-9cb6af54d568 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:35.536991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 561/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:42:35 2022] GET /identity/v3/role_assignments?user.id=70da35d3fa504a9b85acbf4733439894&scope.project.id=f536cf50658943feb4d7b43f2aa81b53&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:37.824382 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8291507-2dfc-48c2-aeb2-7b61bb0d553e None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:37.824961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8291507-2dfc-48c2-aeb2-7b61bb0d553e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:37.824961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8291507-2dfc-48c2-aeb2-7b61bb0d553e None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:37.826247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 601/1162] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:37.835137 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf92ac4-c82b-450b-89e2-67a4d125d5eb None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:37.835658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf92ac4-c82b-450b-89e2-67a4d125d5eb None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:37.835658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf92ac4-c82b-450b-89e2-67a4d125d5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:37.853903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-acf92ac4-c82b-450b-89e2-67a4d125d5eb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:37.855122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-acf92ac4-c82b-450b-89e2-67a4d125d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:37.927150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 562/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:38.197771 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8291507-2dfc-48c2-aeb2-7b61bb0d553e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:38.267661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:38.298179 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8291507-2dfc-48c2-aeb2-7b61bb0d553e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:38.332390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9619e9-4c96-4636-8eb0-bd0553543375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:38.332983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9619e9-4c96-4636-8eb0-bd0553543375 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:38.333281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9619e9-4c96-4636-8eb0-bd0553543375 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:38.333525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9619e9-4c96-4636-8eb0-bd0553543375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:38.399069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9619e9-4c96-4636-8eb0-bd0553543375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:38.400537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9619e9-4c96-4636-8eb0-bd0553543375 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:38.465641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:38.496870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 602/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:42:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:42:38.503016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf92ac4-c82b-450b-89e2-67a4d125d5eb None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:38.551340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:38.599759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf92ac4-c82b-450b-89e2-67a4d125d5eb None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:38.633983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4820d7-e6a8-4bd7-947e-40e5e822867c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:38.634602 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4820d7-e6a8-4bd7-947e-40e5e822867c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:38.634874 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4820d7-e6a8-4bd7-947e-40e5e822867c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:38.635143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4820d7-e6a8-4bd7-947e-40e5e822867c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:38.641808 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4820d7-e6a8-4bd7-947e-40e5e822867c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:38.643235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4820d7-e6a8-4bd7-947e-40e5e822867c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-ff4820d7-e6a8-4bd7-947e-40e5e822867c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:38.654878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:38.656144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:42:38.657245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 563/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 11:42:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:38.662337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9619e9-4c96-4636-8eb0-bd0553543375 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:38.760079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9619e9-4c96-4636-8eb0-bd0553543375 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:38.793966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170d837-229b-444a-8f16-e223b7da6597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:38.794601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170d837-229b-444a-8f16-e223b7da6597 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:38.794878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170d837-229b-444a-8f16-e223b7da6597 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:38.795157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170d837-229b-444a-8f16-e223b7da6597 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:38.795794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170d837-229b-444a-8f16-e223b7da6597 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:38.796091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170d837-229b-444a-8f16-e223b7da6597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:38.797482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170d837-229b-444a-8f16-e223b7da6597 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:38.805596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 603/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Oct 4 11:42:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:40.998256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033ab2-0651-4692-9001-47102ca05397 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:40.998824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033ab2-0651-4692-9001-47102ca05397 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:40.998824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033ab2-0651-4692-9001-47102ca05397 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:41.000148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 564/1167] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:41.005551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b7b3a-3458-4b72-9a13-76e976439386 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:41.005943 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b7b3a-3458-4b72-9a13-76e976439386 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:41.006096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b7b3a-3458-4b72-9a13-76e976439386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:41.024090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e74b7b3a-3458-4b72-9a13-76e976439386 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:41.025298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e74b7b3a-3458-4b72-9a13-76e976439386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:41.051418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:41.092440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 604/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:41.359837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb033ab2-0651-4692-9001-47102ca05397 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:41.462980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb033ab2-0651-4692-9001-47102ca05397 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:41.473918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:41.497550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5f9bc-9019-445f-a36a-2f3c91335406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:41.498151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5f9bc-9019-445f-a36a-2f3c91335406 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:41.498431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5f9bc-9019-445f-a36a-2f3c91335406 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:41.498704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5f9bc-9019-445f-a36a-2f3c91335406 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:41.499364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5f9bc-9019-445f-a36a-2f3c91335406 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:41.500806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5f9bc-9019-445f-a36a-2f3c91335406 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:41.584498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 565/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Tue Oct 4 11:42:41 2022] POST /identity/v3/domains => generated 267 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:42:43.820238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce8d39-fa1a-430d-8ccc-b2e75582b821 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:43.820770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce8d39-fa1a-430d-8ccc-b2e75582b821 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:43.820866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce8d39-fa1a-430d-8ccc-b2e75582b821 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:43.822133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 605/1170] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:43.827473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96bc33-3e48-408c-bede-e3eff0a5d485 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:43.827913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96bc33-3e48-408c-bede-e3eff0a5d485 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:43.828016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96bc33-3e48-408c-bede-e3eff0a5d485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:43.846445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-3c96bc33-3e48-408c-bede-e3eff0a5d485 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:43.847631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3c96bc33-3e48-408c-bede-e3eff0a5d485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:43.919219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 566/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:44.189257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce8d39-fa1a-430d-8ccc-b2e75582b821 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:44.290830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce8d39-fa1a-430d-8ccc-b2e75582b821 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:44.324798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cda1e-4de5-48f4-a392-40f59cf361d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:44.325427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cda1e-4de5-48f4-a392-40f59cf361d2 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:44.325710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cda1e-4de5-48f4-a392-40f59cf361d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:44.325989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cda1e-4de5-48f4-a392-40f59cf361d2 None admin] PATH_INFO: `/v3/domains/1005e21be377409cbd492007cba3affa` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:44.336568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2cda1e-4de5-48f4-a392-40f59cf361d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1005e21be377409cbd492007cba3affa)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:44.338015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2cda1e-4de5-48f4-a392-40f59cf361d2 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:44.339626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 606/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Tue Oct 4 11:42:44 2022] GET /identity/v3/domains/1005e21be377409cbd492007cba3affa => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:44.345474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96bc33-3e48-408c-bede-e3eff0a5d485 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:44.446312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:44.449697 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96bc33-3e48-408c-bede-e3eff0a5d485 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:44.485136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d75de8-3530-4708-a62b-ab3a3b1863c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:44.485737 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d75de8-3530-4708-a62b-ab3a3b1863c2 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:44.486020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d75de8-3530-4708-a62b-ab3a3b1863c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:44.486300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d75de8-3530-4708-a62b-ab3a3b1863c2 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:44.486968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d75de8-3530-4708-a62b-ab3a3b1863c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:44.488436 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d75de8-3530-4708-a62b-ab3a3b1863c2 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:44.611737 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 567/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:42:44 2022] POST /identity/v3/projects => generated 361 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:42:46.842838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc20f76-dcd4-436d-852e-d7ab84fde64a None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:46.843329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc20f76-dcd4-436d-852e-d7ab84fde64a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:46.843440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc20f76-dcd4-436d-852e-d7ab84fde64a None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:46.844766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 607/1174] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:46.850544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ee7b-8260-46c2-832e-f54de98c09f3 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:46.850979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ee7b-8260-46c2-832e-f54de98c09f3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:46.851085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ee7b-8260-46c2-832e-f54de98c09f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:46.869886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-4f22ee7b-8260-46c2-832e-f54de98c09f3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:46.871086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4f22ee7b-8260-46c2-832e-f54de98c09f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:46.938339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 568/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:47.209091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc20f76-dcd4-436d-852e-d7ab84fde64a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:47.235288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:47.315609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc20f76-dcd4-436d-852e-d7ab84fde64a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:47.348247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e10fc2-5309-41ed-8a1d-6b9166264985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:47.349409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e10fc2-5309-41ed-8a1d-6b9166264985 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:47.349720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e10fc2-5309-41ed-8a1d-6b9166264985 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:47.350004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e10fc2-5309-41ed-8a1d-6b9166264985 None admin] PATH_INFO: `/v3/domains/1005e21be377409cbd492007cba3affa` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:47.351150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e10fc2-5309-41ed-8a1d-6b9166264985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1005e21be377409cbd492007cba3affa)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:47.352641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e10fc2-5309-41ed-8a1d-6b9166264985 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:47.354236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 608/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Tue Oct 4 11:42:47 2022] GET /identity/v3/domains/1005e21be377409cbd492007cba3affa => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:47.360450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22ee7b-8260-46c2-832e-f54de98c09f3 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:47.444422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:47.463383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22ee7b-8260-46c2-832e-f54de98c09f3 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:47.497371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a62b85-6d24-403e-a4f4-74468b53c614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:47.497972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a62b85-6d24-403e-a4f4-74468b53c614 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:47.498257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a62b85-6d24-403e-a4f4-74468b53c614 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:47.498534 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a62b85-6d24-403e-a4f4-74468b53c614 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:47.499196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a62b85-6d24-403e-a4f4-74468b53c614 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:47.500744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a62b85-6d24-403e-a4f4-74468b53c614 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:47.611387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 569/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:42:47 2022] POST /identity/v3/users => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:42:49.826204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32d82e-c042-4bb2-beee-d711b06b356a None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:49.826742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32d82e-c042-4bb2-beee-d711b06b356a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:49.826852 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32d82e-c042-4bb2-beee-d711b06b356a None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:49.828207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 609/1178] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:49.833952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6f7c-ae70-4112-b62e-16dccf461234 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:49.834387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6f7c-ae70-4112-b62e-16dccf461234 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:49.834496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6f7c-ae70-4112-b62e-16dccf461234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:49.853678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e29c6f7c-ae70-4112-b62e-16dccf461234 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:49.854891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e29c6f7c-ae70-4112-b62e-16dccf461234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:49.921715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 570/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:50.190247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32d82e-c042-4bb2-beee-d711b06b356a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:50.355997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:50.366143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32d82e-c042-4bb2-beee-d711b06b356a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:50.399799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830cb96-cead-44f5-be38-a152838ecf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:50.400452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830cb96-cead-44f5-be38-a152838ecf82 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:50.400734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830cb96-cead-44f5-be38-a152838ecf82 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:50.401005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830cb96-cead-44f5-be38-a152838ecf82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:50.411538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830cb96-cead-44f5-be38-a152838ecf82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-a830cb96-cead-44f5-be38-a152838ecf82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:42:50.416332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:50.417482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:42:50.418679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 610/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:42:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:50.423564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c6f7c-ae70-4112-b62e-16dccf461234 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:50.487689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:50.525035 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c6f7c-ae70-4112-b62e-16dccf461234 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:50.560360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc09b3-265c-45c0-88c6-f401b9abd245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:50.560964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc09b3-265c-45c0-88c6-f401b9abd245 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:50.561289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc09b3-265c-45c0-88c6-f401b9abd245 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:50.561510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc09b3-265c-45c0-88c6-f401b9abd245 None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:50.562165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc09b3-265c-45c0-88c6-f401b9abd245 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:50.562465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc09b3-265c-45c0-88c6-f401b9abd245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:50.563830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc09b3-265c-45c0-88c6-f401b9abd245 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:50.572286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 571/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:42:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:42:50.578468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830cb96-cead-44f5-be38-a152838ecf82 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:50.678230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830cb96-cead-44f5-be38-a152838ecf82 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:50.712331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed8e49-7a9c-407c-a761-0a8ef1dbb36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:50.712958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed8e49-7a9c-407c-a761-0a8ef1dbb36b None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:50.713237 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed8e49-7a9c-407c-a761-0a8ef1dbb36b None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:50.713508 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed8e49-7a9c-407c-a761-0a8ef1dbb36b None admin] PATH_INFO: `/v3/users/78108db7cebc47598031459b5cb6a1d2` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:50.727085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed8e49-7a9c-407c-a761-0a8ef1dbb36b None admin] RBAC: Authorizing `identity:get_user(user_id=78108db7cebc47598031459b5cb6a1d2)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:50.728566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed8e49-7a9c-407c-a761-0a8ef1dbb36b None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:50.730262 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 611/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:50 2022] GET /identity/v3/users/78108db7cebc47598031459b5cb6a1d2 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:50.736416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc09b3-265c-45c0-88c6-f401b9abd245 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:50.838152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc09b3-265c-45c0-88c6-f401b9abd245 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:50.866471 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:50.872143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ec1c0-0dd9-4f9e-bfa2-48f6dc2fc3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:50.872771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec1c0-0dd9-4f9e-bfa2-48f6dc2fc3e5 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:50.873053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec1c0-0dd9-4f9e-bfa2-48f6dc2fc3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:50.873383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec1c0-0dd9-4f9e-bfa2-48f6dc2fc3e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:50.940038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ec1c0-0dd9-4f9e-bfa2-48f6dc2fc3e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:50.941517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ec1c0-0dd9-4f9e-bfa2-48f6dc2fc3e5 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:51.046213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 572/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:42:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:42:51.052495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed8e49-7a9c-407c-a761-0a8ef1dbb36b None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:51.064630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:51.150381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed8e49-7a9c-407c-a761-0a8ef1dbb36b None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:51.184646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a49f8e-1fbd-42dd-ba7a-305457d11d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:51.185269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a49f8e-1fbd-42dd-ba7a-305457d11d69 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:51.185550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a49f8e-1fbd-42dd-ba7a-305457d11d69 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:51.185825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a49f8e-1fbd-42dd-ba7a-305457d11d69 None admin] PATH_INFO: `/v3/projects/503cfc84c313474699066b848fa7bc47` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:51.186940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a49f8e-1fbd-42dd-ba7a-305457d11d69 None admin] RBAC: Authorizing `identity:get_project(project_id=503cfc84c313474699066b848fa7bc47)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:51.188397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a49f8e-1fbd-42dd-ba7a-305457d11d69 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:51.190022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 612/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:51 2022] GET /identity/v3/projects/503cfc84c313474699066b848fa7bc47 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:51.196799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ec1c0-0dd9-4f9e-bfa2-48f6dc2fc3e5 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:51.226994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:51.299674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ec1c0-0dd9-4f9e-bfa2-48f6dc2fc3e5 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:51.334609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a3c78-58a8-42e5-bc37-17d4beac2b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:51.335116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a3c78-58a8-42e5-bc37-17d4beac2b6e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:51.335389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a3c78-58a8-42e5-bc37-17d4beac2b6e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:51.335658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a3c78-58a8-42e5-bc37-17d4beac2b6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:51.336393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a3c78-58a8-42e5-bc37-17d4beac2b6e None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=503cfc84c313474699066b848fa7bc47, user.id=78108db7cebc47598031459b5cb6a1d2) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:51.336698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a3c78-58a8-42e5-bc37-17d4beac2b6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:51.338115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a3c78-58a8-42e5-bc37-17d4beac2b6e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:51.343545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 573/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:42:51 2022] GET /identity/v3/role_assignments?user.id=78108db7cebc47598031459b5cb6a1d2&scope.project.id=503cfc84c313474699066b848fa7bc47&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:53.491809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3212f13-7be8-486a-8427-780018272d6f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:53.492412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3212f13-7be8-486a-8427-780018272d6f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:53.492412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3212f13-7be8-486a-8427-780018272d6f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:53.493706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 613/1186] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:53.499275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d50f35-2a02-4eda-b8bc-84eeb77790ab None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:53.499807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d50f35-2a02-4eda-b8bc-84eeb77790ab None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:53.499807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d50f35-2a02-4eda-b8bc-84eeb77790ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:53.518713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b3d50f35-2a02-4eda-b8bc-84eeb77790ab None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:53.519951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b3d50f35-2a02-4eda-b8bc-84eeb77790ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:53.590046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 574/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:42:53.868394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3212f13-7be8-486a-8427-780018272d6f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:53.952066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:53.975267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3212f13-7be8-486a-8427-780018272d6f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:54.009278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647edb-f2c8-445a-afdf-1b68ebe784f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:54.009903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647edb-f2c8-445a-afdf-1b68ebe784f7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:54.010179 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647edb-f2c8-445a-afdf-1b68ebe784f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:54.010450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647edb-f2c8-445a-afdf-1b68ebe784f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:54.021197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41647edb-f2c8-445a-afdf-1b68ebe784f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-41647edb-f2c8-445a-afdf-1b68ebe784f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:42:54.025991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:54.027132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:42:54.027984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 614/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:42:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:54.033427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d50f35-2a02-4eda-b8bc-84eeb77790ab None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:54.059130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:54.139647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d50f35-2a02-4eda-b8bc-84eeb77790ab None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:54.177221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44be4e5-fd4e-4701-9e3e-ec4090c7da0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:54.177829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44be4e5-fd4e-4701-9e3e-ec4090c7da0a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:54.178109 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44be4e5-fd4e-4701-9e3e-ec4090c7da0a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:54.178380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44be4e5-fd4e-4701-9e3e-ec4090c7da0a None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:54.179044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44be4e5-fd4e-4701-9e3e-ec4090c7da0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:54.179374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44be4e5-fd4e-4701-9e3e-ec4090c7da0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:54.180896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44be4e5-fd4e-4701-9e3e-ec4090c7da0a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:54.189683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 575/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:42:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:54.196194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647edb-f2c8-445a-afdf-1b68ebe784f7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:54.297715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647edb-f2c8-445a-afdf-1b68ebe784f7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:54.331613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d9541-c04b-4803-a3ef-7e28e071b0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:54.332197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d9541-c04b-4803-a3ef-7e28e071b0f2 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:54.332473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d9541-c04b-4803-a3ef-7e28e071b0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:54.333524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d9541-c04b-4803-a3ef-7e28e071b0f2 None admin] PATH_INFO: `/v3/users/78108db7cebc47598031459b5cb6a1d2` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:54.336412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d9541-c04b-4803-a3ef-7e28e071b0f2 None admin] RBAC: Authorizing `identity:get_user(user_id=78108db7cebc47598031459b5cb6a1d2)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:54.337844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d9541-c04b-4803-a3ef-7e28e071b0f2 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:54.340230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 615/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:54 2022] GET /identity/v3/users/78108db7cebc47598031459b5cb6a1d2 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:54.346114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44be4e5-fd4e-4701-9e3e-ec4090c7da0a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:54.431757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:54.446814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44be4e5-fd4e-4701-9e3e-ec4090c7da0a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:54.481925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f30aa-f300-42b0-b13f-59d22ff15cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:54.482627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f30aa-f300-42b0-b13f-59d22ff15cc0 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:54.483167 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f30aa-f300-42b0-b13f-59d22ff15cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:54.483444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f30aa-f300-42b0-b13f-59d22ff15cc0 None admin] PATH_INFO: `/v3/projects/503cfc84c313474699066b848fa7bc47` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:54.484623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f30aa-f300-42b0-b13f-59d22ff15cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=503cfc84c313474699066b848fa7bc47)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:54.486071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f30aa-f300-42b0-b13f-59d22ff15cc0 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:54.487723 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 576/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:54 2022] GET /identity/v3/projects/503cfc84c313474699066b848fa7bc47 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:54.493835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d9541-c04b-4803-a3ef-7e28e071b0f2 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:54.503412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:54.595075 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d9541-c04b-4803-a3ef-7e28e071b0f2 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:54.630107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d62f82-de79-49f5-9263-43466ed89724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:54.630765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62f82-de79-49f5-9263-43466ed89724 None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:54.631041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62f82-de79-49f5-9263-43466ed89724 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:54.631311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62f82-de79-49f5-9263-43466ed89724 None admin] PATH_INFO: `/v3/projects/503cfc84c313474699066b848fa7bc47/users/78108db7cebc47598031459b5cb6a1d2/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:54.633212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d62f82-de79-49f5-9263-43466ed89724 None admin] RBAC: Authorizing `identity:create_grant(project_id=503cfc84c313474699066b848fa7bc47, user_id=78108db7cebc47598031459b5cb6a1d2, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:54.634630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d62f82-de79-49f5-9263-43466ed89724 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:54.708841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 616/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:42:54 2022] PUT /identity/v3/projects/503cfc84c313474699066b848fa7bc47/users/78108db7cebc47598031459b5cb6a1d2/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:42:56.874532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e08c1-22cd-4eca-b4e6-e58a11ef3bc3 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:56.875064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e08c1-22cd-4eca-b4e6-e58a11ef3bc3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:56.875064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e08c1-22cd-4eca-b4e6-e58a11ef3bc3 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:56.876459 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 577/1193] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:42:56.881770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ff7a5-1fab-4a85-94a2-1573fb44e486 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:56.882202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ff7a5-1fab-4a85-94a2-1573fb44e486 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:56.882354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ff7a5-1fab-4a85-94a2-1573fb44e486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:56.901166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f05ff7a5-1fab-4a85-94a2-1573fb44e486 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:42:56.902395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f05ff7a5-1fab-4a85-94a2-1573fb44e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:42:56.963161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:56.971013 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 617/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:42:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:42:57.240994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e08c1-22cd-4eca-b4e6-e58a11ef3bc3 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:57.343197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e08c1-22cd-4eca-b4e6-e58a11ef3bc3 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:57.375560 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4769d-9fee-4dd0-9931-58a82a73188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:57.376197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4769d-9fee-4dd0-9931-58a82a73188a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:57.376474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4769d-9fee-4dd0-9931-58a82a73188a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:57.376744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4769d-9fee-4dd0-9931-58a82a73188a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:57.388647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4769d-9fee-4dd0-9931-58a82a73188a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-d8e4769d-9fee-4dd0-9931-58a82a73188a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:42:57.393397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:42:57.394482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:42:57.396116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 578/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:42:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:42:57.401406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ff7a5-1fab-4a85-94a2-1573fb44e486 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:57.496076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ff7a5-1fab-4a85-94a2-1573fb44e486 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:57.529193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ff55f-9260-4cc9-a15b-a0af1ee7e35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:57.529870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ff55f-9260-4cc9-a15b-a0af1ee7e35a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:57.530154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ff55f-9260-4cc9-a15b-a0af1ee7e35a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:57.530429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ff55f-9260-4cc9-a15b-a0af1ee7e35a None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:57.531085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ff55f-9260-4cc9-a15b-a0af1ee7e35a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:57.531386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ff55f-9260-4cc9-a15b-a0af1ee7e35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:57.532781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ff55f-9260-4cc9-a15b-a0af1ee7e35a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:57.540713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 618/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:57.546910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4769d-9fee-4dd0-9931-58a82a73188a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:57.563387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:57.647420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4769d-9fee-4dd0-9931-58a82a73188a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:57.688395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa7216-c7e4-4ae6-927b-8e675052b590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:57.689037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa7216-c7e4-4ae6-927b-8e675052b590 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:57.689321 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa7216-c7e4-4ae6-927b-8e675052b590 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:57.689598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa7216-c7e4-4ae6-927b-8e675052b590 None admin] PATH_INFO: `/v3/users/78108db7cebc47598031459b5cb6a1d2` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:57.690747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa7216-c7e4-4ae6-927b-8e675052b590 None admin] RBAC: Authorizing `identity:get_user(user_id=78108db7cebc47598031459b5cb6a1d2)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:57.692832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa7216-c7e4-4ae6-927b-8e675052b590 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:57.694444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 579/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Oct 4 11:42:57 2022] GET /identity/v3/users/78108db7cebc47598031459b5cb6a1d2 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:57.700514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ff55f-9260-4cc9-a15b-a0af1ee7e35a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:57.759609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:57.804561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ff55f-9260-4cc9-a15b-a0af1ee7e35a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:57.837905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81314c-3d59-44fd-9594-b80a5e63b995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:57.838505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81314c-3d59-44fd-9594-b80a5e63b995 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:57.838782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81314c-3d59-44fd-9594-b80a5e63b995 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:57.839051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81314c-3d59-44fd-9594-b80a5e63b995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:57.904954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81314c-3d59-44fd-9594-b80a5e63b995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:57.906399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81314c-3d59-44fd-9594-b80a5e63b995 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:57.961592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:58.004258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 619/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:42:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:42:58.010722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa7216-c7e4-4ae6-927b-8e675052b590 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:58.103059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:42:58.122796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa7216-c7e4-4ae6-927b-8e675052b590 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:58.158195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5606144-0a33-4f38-9f10-078af678e669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:58.158849 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5606144-0a33-4f38-9f10-078af678e669 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:58.159105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5606144-0a33-4f38-9f10-078af678e669 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:58.159376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5606144-0a33-4f38-9f10-078af678e669 None admin] PATH_INFO: `/v3/projects/503cfc84c313474699066b848fa7bc47` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:58.161056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5606144-0a33-4f38-9f10-078af678e669 None admin] RBAC: Authorizing `identity:get_project(project_id=503cfc84c313474699066b848fa7bc47)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:58.162490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5606144-0a33-4f38-9f10-078af678e669 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:58.164125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 580/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Oct 4 11:42:58 2022] GET /identity/v3/projects/503cfc84c313474699066b848fa7bc47 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:42:58.170328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81314c-3d59-44fd-9594-b80a5e63b995 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:42:58.268358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81314c-3d59-44fd-9594-b80a5e63b995 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:42:58.303065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef03a6a-1944-4619-ae61-b4b65f7d25c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:42:58.303672 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef03a6a-1944-4619-ae61-b4b65f7d25c0 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:42:58.303955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef03a6a-1944-4619-ae61-b4b65f7d25c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:42:58.304256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef03a6a-1944-4619-ae61-b4b65f7d25c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:42:58.304983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef03a6a-1944-4619-ae61-b4b65f7d25c0 None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=503cfc84c313474699066b848fa7bc47, user.id=78108db7cebc47598031459b5cb6a1d2) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:42:58.305290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef03a6a-1944-4619-ae61-b4b65f7d25c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:42:58.306753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef03a6a-1944-4619-ae61-b4b65f7d25c0 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:42:58.312546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 620/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:42:58 2022] GET /identity/v3/role_assignments?user.id=78108db7cebc47598031459b5cb6a1d2&scope.project.id=503cfc84c313474699066b848fa7bc47&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:43:08.153745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=25 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:43:08.309925 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:17.963222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dc8bd-b5d2-4232-a2fb-e91dba994820 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:17.963813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dc8bd-b5d2-4232-a2fb-e91dba994820 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:17.963813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dc8bd-b5d2-4232-a2fb-e91dba994820 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:17.965147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 581/1201] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:44:17.970397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a10d3-783f-4237-a2c7-9ad58141d73d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:17.970845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a10d3-783f-4237-a2c7-9ad58141d73d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:17.970940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a10d3-783f-4237-a2c7-9ad58141d73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:17.989598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e10a10d3-783f-4237-a2c7-9ad58141d73d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:17.990774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e10a10d3-783f-4237-a2c7-9ad58141d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:18.055644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 621/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:44:18.330320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dc8bd-b5d2-4232-a2fb-e91dba994820 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:18.433207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dc8bd-b5d2-4232-a2fb-e91dba994820 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:18.467139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d788c2d-4b43-4945-a1b3-fde370b71184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:18.467766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d788c2d-4b43-4945-a1b3-fde370b71184 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:18.468042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d788c2d-4b43-4945-a1b3-fde370b71184 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:18.468340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d788c2d-4b43-4945-a1b3-fde370b71184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:18.479790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d788c2d-4b43-4945-a1b3-fde370b71184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:18.481302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d788c2d-4b43-4945-a1b3-fde370b71184 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-4d788c2d-4b43-4945-a1b3-fde370b71184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:18.497361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:18.498679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:18.499909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:18.499909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:18.499909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:18.499909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 582/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:18.505128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a10d3-783f-4237-a2c7-9ad58141d73d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:18.606729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a10d3-783f-4237-a2c7-9ad58141d73d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:18.634679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:18.640567 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942ea16-5ad2-4cbc-8a1c-45f99d6676c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:18.641163 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942ea16-5ad2-4cbc-8a1c-45f99d6676c8 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:18.642000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942ea16-5ad2-4cbc-8a1c-45f99d6676c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:18.642270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942ea16-5ad2-4cbc-8a1c-45f99d6676c8 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:18.642899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942ea16-5ad2-4cbc-8a1c-45f99d6676c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:18.643197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942ea16-5ad2-4cbc-8a1c-45f99d6676c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:18.644570 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942ea16-5ad2-4cbc-8a1c-45f99d6676c8 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:18.655725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 622/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:18.662072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d788c2d-4b43-4945-a1b3-fde370b71184 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:18.716904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:18.764293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d788c2d-4b43-4945-a1b3-fde370b71184 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:18.801951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5b355-7510-460c-bdcc-dbebd5dfd5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:18.801951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5b355-7510-460c-bdcc-dbebd5dfd5af None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:18.801951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5b355-7510-460c-bdcc-dbebd5dfd5af None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:18.801951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5b355-7510-460c-bdcc-dbebd5dfd5af None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:18.802384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5b355-7510-460c-bdcc-dbebd5dfd5af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:18.802960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5b355-7510-460c-bdcc-dbebd5dfd5af None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:18.989062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 583/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Oct 4 11:44:18 2022] POST /identity/v3/users => generated 251 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:44:21.191286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218db34-92c6-4c1d-8d0e-0bc58f37cf9d None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:21.191823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218db34-92c6-4c1d-8d0e-0bc58f37cf9d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:21.191823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218db34-92c6-4c1d-8d0e-0bc58f37cf9d None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:21.193217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 623/1206] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:44:21.198974 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c09bd2-01d5-4309-afcb-02d0767038ea None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:21.200442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c09bd2-01d5-4309-afcb-02d0767038ea None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:21.200442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c09bd2-01d5-4309-afcb-02d0767038ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:21.218237 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d8c09bd2-01d5-4309-afcb-02d0767038ea None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:21.220050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d8c09bd2-01d5-4309-afcb-02d0767038ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:21.289772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 584/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:44:21.565437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218db34-92c6-4c1d-8d0e-0bc58f37cf9d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:21.667826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218db34-92c6-4c1d-8d0e-0bc58f37cf9d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:21.702877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c13cc-77f0-4e3e-9d0a-5f43195686e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:21.703503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c13cc-77f0-4e3e-9d0a-5f43195686e3 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:21.704330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c13cc-77f0-4e3e-9d0a-5f43195686e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:21.704604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c13cc-77f0-4e3e-9d0a-5f43195686e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:21.716127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c13cc-77f0-4e3e-9d0a-5f43195686e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-216c13cc-77f0-4e3e-9d0a-5f43195686e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:44:21.721528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:21.722615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:21.723842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 624/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:44:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:21.728877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c09bd2-01d5-4309-afcb-02d0767038ea None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:21.769489 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:21.830330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c09bd2-01d5-4309-afcb-02d0767038ea None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:21.864409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf2273-4b52-40b6-94ba-0ebd21e0621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:21.865008 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf2273-4b52-40b6-94ba-0ebd21e0621a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:21.865287 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf2273-4b52-40b6-94ba-0ebd21e0621a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:21.865559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf2273-4b52-40b6-94ba-0ebd21e0621a None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:21.866222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf2273-4b52-40b6-94ba-0ebd21e0621a None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:21.866522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf2273-4b52-40b6-94ba-0ebd21e0621a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:21.867879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf2273-4b52-40b6-94ba-0ebd21e0621a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:21.878792 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 585/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:44:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:21.885007 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c13cc-77f0-4e3e-9d0a-5f43195686e3 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:21.936143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:21.986587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c13cc-77f0-4e3e-9d0a-5f43195686e3 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:22.023769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15377ab4-83cf-42ca-b18e-f3bc504b4116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:22.024429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15377ab4-83cf-42ca-b18e-f3bc504b4116 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:22.024712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15377ab4-83cf-42ca-b18e-f3bc504b4116 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:22.024990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15377ab4-83cf-42ca-b18e-f3bc504b4116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:22.035338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15377ab4-83cf-42ca-b18e-f3bc504b4116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:22.036818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15377ab4-83cf-42ca-b18e-f3bc504b4116 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-15377ab4-83cf-42ca-b18e-f3bc504b4116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:22.051220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:22.053274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:22.053274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:22.053274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:22.053274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 625/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:44:22.058480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf2273-4b52-40b6-94ba-0ebd21e0621a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:22.165798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:22.168732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf2273-4b52-40b6-94ba-0ebd21e0621a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:22.202462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac8a9-a489-43f7-98ce-38dcb77bf1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:22.203230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac8a9-a489-43f7-98ce-38dcb77bf1d3 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:22.203350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac8a9-a489-43f7-98ce-38dcb77bf1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:22.203575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac8a9-a489-43f7-98ce-38dcb77bf1d3 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:22.204880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac8a9-a489-43f7-98ce-38dcb77bf1d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:22.205183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac8a9-a489-43f7-98ce-38dcb77bf1d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:22.206536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac8a9-a489-43f7-98ce-38dcb77bf1d3 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:22.218196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 586/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:22.223917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15377ab4-83cf-42ca-b18e-f3bc504b4116 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:22.323336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15377ab4-83cf-42ca-b18e-f3bc504b4116 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:22.324884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:22.356125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75077a9e-1635-4f97-aab9-1a2f8d1397a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:22.361895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75077a9e-1635-4f97-aab9-1a2f8d1397a1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:22.361895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75077a9e-1635-4f97-aab9-1a2f8d1397a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:22.361895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75077a9e-1635-4f97-aab9-1a2f8d1397a1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:22.365638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75077a9e-1635-4f97-aab9-1a2f8d1397a1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:22.367064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75077a9e-1635-4f97-aab9-1a2f8d1397a1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-75077a9e-1635-4f97-aab9-1a2f8d1397a1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:44:22.378540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:22.379741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:22.380856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 626/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:22.385669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac8a9-a489-43f7-98ce-38dcb77bf1d3 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:22.489354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac8a9-a489-43f7-98ce-38dcb77bf1d3 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:22.524499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc15c8c-f7ff-48f4-acee-91fe6f455a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:22.525097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc15c8c-f7ff-48f4-acee-91fe6f455a1a None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:22.525377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc15c8c-f7ff-48f4-acee-91fe6f455a1a None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:22.525650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc15c8c-f7ff-48f4-acee-91fe6f455a1a None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:22.526368 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc15c8c-f7ff-48f4-acee-91fe6f455a1a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:22.526671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc15c8c-f7ff-48f4-acee-91fe6f455a1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:22.528234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc15c8c-f7ff-48f4-acee-91fe6f455a1a None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:22.546339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 587/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 11:44:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:22.552688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75077a9e-1635-4f97-aab9-1a2f8d1397a1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:22.653326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75077a9e-1635-4f97-aab9-1a2f8d1397a1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:22.686854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182b5ff-2779-4379-871c-2e441cbf5aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:22.687470 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182b5ff-2779-4379-871c-2e441cbf5aec None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:22.687760 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182b5ff-2779-4379-871c-2e441cbf5aec None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:22.688036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182b5ff-2779-4379-871c-2e441cbf5aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:22.720121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:22.758871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182b5ff-2779-4379-871c-2e441cbf5aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:22.760261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182b5ff-2779-4379-871c-2e441cbf5aec None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:22.858584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 627/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:44:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:44:22.865260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc15c8c-f7ff-48f4-acee-91fe6f455a1a None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:22.884920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:22.964405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc15c8c-f7ff-48f4-acee-91fe6f455a1a None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:22.998063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e2d0f-7e5c-4928-98fe-364fa9588844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:22.998676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e2d0f-7e5c-4928-98fe-364fa9588844 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:22.998950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e2d0f-7e5c-4928-98fe-364fa9588844 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:22.999217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e2d0f-7e5c-4928-98fe-364fa9588844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:23.014966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e2d0f-7e5c-4928-98fe-364fa9588844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:23.017168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e2d0f-7e5c-4928-98fe-364fa9588844 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-a06e2d0f-7e5c-4928-98fe-364fa9588844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:23.032801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:23.033959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:23.035077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:23.035077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:23.035077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:23.035077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 588/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:23.039958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182b5ff-2779-4379-871c-2e441cbf5aec None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:23.101241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:23.140411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182b5ff-2779-4379-871c-2e441cbf5aec None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:23.175088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd584b-d7db-430e-a8c8-abaa2cc80433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:23.175701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd584b-d7db-430e-a8c8-abaa2cc80433 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:23.175982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd584b-d7db-430e-a8c8-abaa2cc80433 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:23.176307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd584b-d7db-430e-a8c8-abaa2cc80433 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:23.177037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd584b-d7db-430e-a8c8-abaa2cc80433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:23.177375 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd584b-d7db-430e-a8c8-abaa2cc80433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:23.179082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd584b-d7db-430e-a8c8-abaa2cc80433 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:23.191167 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 628/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:23.197290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e2d0f-7e5c-4928-98fe-364fa9588844 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:23.267407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:23.297703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e2d0f-7e5c-4928-98fe-364fa9588844 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:23.330802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520c60a-b673-4341-a75e-5789d45b48d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:23.331543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520c60a-b673-4341-a75e-5789d45b48d7 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:23.331826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520c60a-b673-4341-a75e-5789d45b48d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:23.332097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520c60a-b673-4341-a75e-5789d45b48d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:23.338919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520c60a-b673-4341-a75e-5789d45b48d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:23.340501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520c60a-b673-4341-a75e-5789d45b48d7 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-f520c60a-b673-4341-a75e-5789d45b48d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:44:23.352948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:23.359183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:23.360405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:44:23.360405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:44:23.360405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:44:23.360405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:44:23.360405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:44:23.360405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:44:23.360405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:23.360405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:23.360405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 589/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:44:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:23.361044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd584b-d7db-430e-a8c8-abaa2cc80433 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:23.459806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd584b-d7db-430e-a8c8-abaa2cc80433 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:23.481702 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:23.493404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41021ef0-6fb4-4776-8649-d56c6cf9264d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:23.494008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41021ef0-6fb4-4776-8649-d56c6cf9264d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:23.494283 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41021ef0-6fb4-4776-8649-d56c6cf9264d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:23.494551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41021ef0-6fb4-4776-8649-d56c6cf9264d None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:23.495232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41021ef0-6fb4-4776-8649-d56c6cf9264d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:23.495532 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41021ef0-6fb4-4776-8649-d56c6cf9264d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:23.496975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41021ef0-6fb4-4776-8649-d56c6cf9264d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:23.510449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 629/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:44:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:23.517360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520c60a-b673-4341-a75e-5789d45b48d7 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:23.618492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520c60a-b673-4341-a75e-5789d45b48d7 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:23.647124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:23.653017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7489a4b-2258-4efe-a6b4-a05e17b73e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:23.653723 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7489a4b-2258-4efe-a6b4-a05e17b73e0e None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:23.653999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7489a4b-2258-4efe-a6b4-a05e17b73e0e None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:23.654270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7489a4b-2258-4efe-a6b4-a05e17b73e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:23.654992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7489a4b-2258-4efe-a6b4-a05e17b73e0e None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=16374dbb6d084ddfa3b49afd19c7c325) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:23.655302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7489a4b-2258-4efe-a6b4-a05e17b73e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:23.656806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7489a4b-2258-4efe-a6b4-a05e17b73e0e None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:23.662314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 590/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:44:23 2022] GET /identity/v3/role_assignments?user.id=16374dbb6d084ddfa3b49afd19c7c325&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:44:25.805642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11925acc-a5ef-47de-9000-b12f66514e0c None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:25.806298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11925acc-a5ef-47de-9000-b12f66514e0c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:25.806298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11925acc-a5ef-47de-9000-b12f66514e0c None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:25.807579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 630/1220] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:44:25.813737 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb80ad-ac05-4684-a2e1-24b5de8e8156 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:25.814357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb80ad-ac05-4684-a2e1-24b5de8e8156 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:25.814463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb80ad-ac05-4684-a2e1-24b5de8e8156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:25.832660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-91cb80ad-ac05-4684-a2e1-24b5de8e8156 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:25.833835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-91cb80ad-ac05-4684-a2e1-24b5de8e8156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:25.903094 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 591/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:44:26.175212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11925acc-a5ef-47de-9000-b12f66514e0c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:26.278756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11925acc-a5ef-47de-9000-b12f66514e0c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:26.314974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1374e84-2880-4f11-b8eb-5481f3ccaf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:26.315598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1374e84-2880-4f11-b8eb-5481f3ccaf61 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:26.315881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1374e84-2880-4f11-b8eb-5481f3ccaf61 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:26.316156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1374e84-2880-4f11-b8eb-5481f3ccaf61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:26.326538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1374e84-2880-4f11-b8eb-5481f3ccaf61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-f1374e84-2880-4f11-b8eb-5481f3ccaf61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:44:26.331501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:26.332823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:26.333766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 631/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:44:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:26.338769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb80ad-ac05-4684-a2e1-24b5de8e8156 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:26.440061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb80ad-ac05-4684-a2e1-24b5de8e8156 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:26.463726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:26.474825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb0269-8eae-4158-af4d-565088bf3eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:26.475431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb0269-8eae-4158-af4d-565088bf3eca None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:26.475712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb0269-8eae-4158-af4d-565088bf3eca None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:26.476214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb0269-8eae-4158-af4d-565088bf3eca None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:26.476912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb0269-8eae-4158-af4d-565088bf3eca None admin] RBAC: Adding query filter params (name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:26.477217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb0269-8eae-4158-af4d-565088bf3eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:26.478584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb0269-8eae-4158-af4d-565088bf3eca None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:26.487407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 592/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:44:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:26.493288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1374e84-2880-4f11-b8eb-5481f3ccaf61 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:26.536243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:26.589978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1374e84-2880-4f11-b8eb-5481f3ccaf61 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:26.625914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce456d-3f98-41bb-ae0b-8ad6cfa6eced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:26.626576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce456d-3f98-41bb-ae0b-8ad6cfa6eced None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:26.626858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce456d-3f98-41bb-ae0b-8ad6cfa6eced None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:26.627145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce456d-3f98-41bb-ae0b-8ad6cfa6eced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:26.637754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce456d-3f98-41bb-ae0b-8ad6cfa6eced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:26.639207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce456d-3f98-41bb-ae0b-8ad6cfa6eced None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-b1ce456d-3f98-41bb-ae0b-8ad6cfa6eced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:26.654251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:26.655461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:26.656816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:26.656816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:26.656816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:26.656816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 632/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:44:26.661712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb0269-8eae-4158-af4d-565088bf3eca None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:26.764144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb0269-8eae-4158-af4d-565088bf3eca None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:26.798581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3744a9-ca71-43eb-ae74-01ff9e6a5f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:26.799193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3744a9-ca71-43eb-ae74-01ff9e6a5f7b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:26.799918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3744a9-ca71-43eb-ae74-01ff9e6a5f7b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:26.799918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3744a9-ca71-43eb-ae74-01ff9e6a5f7b None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:26.800381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3744a9-ca71-43eb-ae74-01ff9e6a5f7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:26.800682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3744a9-ca71-43eb-ae74-01ff9e6a5f7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:26.802058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3744a9-ca71-43eb-ae74-01ff9e6a5f7b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:26.814075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 593/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:26.819769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce456d-3f98-41bb-ae0b-8ad6cfa6eced None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:26.912964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:26.919812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce456d-3f98-41bb-ae0b-8ad6cfa6eced None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:26.959951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f4d9a-dc86-4b4e-ab9b-6e0584885031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:26.960609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f4d9a-dc86-4b4e-ab9b-6e0584885031 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:26.960885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f4d9a-dc86-4b4e-ab9b-6e0584885031 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:26.961155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f4d9a-dc86-4b4e-ab9b-6e0584885031 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:26.968746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f4d9a-dc86-4b4e-ab9b-6e0584885031 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:26.976209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f4d9a-dc86-4b4e-ab9b-6e0584885031 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-fc1f4d9a-dc86-4b4e-ab9b-6e0584885031 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:44:26.981280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:26.982859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:26.984234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 633/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:26.989080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3744a9-ca71-43eb-ae74-01ff9e6a5f7b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:27.049179 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:27.092564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3744a9-ca71-43eb-ae74-01ff9e6a5f7b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:27.129167 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce412585-0a7b-496b-9979-25f464abdea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:27.129770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce412585-0a7b-496b-9979-25f464abdea1 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:27.130052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce412585-0a7b-496b-9979-25f464abdea1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:27.130359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce412585-0a7b-496b-9979-25f464abdea1 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:27.131085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce412585-0a7b-496b-9979-25f464abdea1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:27.131395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce412585-0a7b-496b-9979-25f464abdea1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:27.132860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce412585-0a7b-496b-9979-25f464abdea1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:27.150072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 594/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 11:44:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:27.156625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f4d9a-dc86-4b4e-ab9b-6e0584885031 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:27.261525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f4d9a-dc86-4b4e-ab9b-6e0584885031 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:27.296083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd8275-c058-460b-bf88-02d8c127052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:27.296748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd8275-c058-460b-bf88-02d8c127052f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:27.297029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd8275-c058-460b-bf88-02d8c127052f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:27.297303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd8275-c058-460b-bf88-02d8c127052f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:27.308761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd8275-c058-460b-bf88-02d8c127052f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:27.310136 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd8275-c058-460b-bf88-02d8c127052f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-63fd8275-c058-460b-bf88-02d8c127052f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:27.324758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:27.330750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:27.331939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:27.331939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:27.331939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:27.331939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 634/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:27.332319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce412585-0a7b-496b-9979-25f464abdea1 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:27.431423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce412585-0a7b-496b-9979-25f464abdea1 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:27.433756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:27.464142 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbd2b7-13ab-4b98-835a-cf85c469c430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:27.464757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbd2b7-13ab-4b98-835a-cf85c469c430 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:27.465033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbd2b7-13ab-4b98-835a-cf85c469c430 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:27.465300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbd2b7-13ab-4b98-835a-cf85c469c430 None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:27.465927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbd2b7-13ab-4b98-835a-cf85c469c430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:27.466224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbd2b7-13ab-4b98-835a-cf85c469c430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:27.467685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbd2b7-13ab-4b98-835a-cf85c469c430 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:27.479081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 595/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:27.486022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd8275-c058-460b-bf88-02d8c127052f None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:27.495524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:27.591702 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd8275-c058-460b-bf88-02d8c127052f None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:27.624712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdf536-3a8a-44d1-b444-830c662fe763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:27.625333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdf536-3a8a-44d1-b444-830c662fe763 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:27.625607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdf536-3a8a-44d1-b444-830c662fe763 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:27.625905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdf536-3a8a-44d1-b444-830c662fe763 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:27.632381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdf536-3a8a-44d1-b444-830c662fe763 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:27.633815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdf536-3a8a-44d1-b444-830c662fe763 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-becdf536-3a8a-44d1-b444-830c662fe763 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:44:27.645410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:27.646542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:27.647685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:44:27.647685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:44:27.647685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:44:27.647685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:44:27.647685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:44:27.647685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:44:27.647685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:27.647685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:27.647685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 635/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:44:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:27.652380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbd2b7-13ab-4b98-835a-cf85c469c430 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:27.763561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbd2b7-13ab-4b98-835a-cf85c469c430 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:27.804673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac041d-bf6f-459c-bf7b-101fc5aa922d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:27.808646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac041d-bf6f-459c-bf7b-101fc5aa922d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:27.808646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac041d-bf6f-459c-bf7b-101fc5aa922d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:27.808646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac041d-bf6f-459c-bf7b-101fc5aa922d None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:27.808646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac041d-bf6f-459c-bf7b-101fc5aa922d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:27.808646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac041d-bf6f-459c-bf7b-101fc5aa922d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:27.808646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac041d-bf6f-459c-bf7b-101fc5aa922d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:27.820953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 596/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:44:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:27.826970 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdf536-3a8a-44d1-b444-830c662fe763 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:27.893159 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:27.929087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdf536-3a8a-44d1-b444-830c662fe763 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:27.962664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af05bf-3c24-4896-b4cb-0b1627fd411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:27.963418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af05bf-3c24-4896-b4cb-0b1627fd411d None admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:27.963700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af05bf-3c24-4896-b4cb-0b1627fd411d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:27.963979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af05bf-3c24-4896-b4cb-0b1627fd411d None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/16374dbb6d084ddfa3b49afd19c7c325/roles/5b919ba5900449358b3277530a7eff6c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:27.977957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af05bf-3c24-4896-b4cb-0b1627fd411d None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=16374dbb6d084ddfa3b49afd19c7c325, role_id=5b919ba5900449358b3277530a7eff6c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:27.979381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af05bf-3c24-4896-b4cb-0b1627fd411d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:28.123575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 636/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:44:27 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/16374dbb6d084ddfa3b49afd19c7c325/roles/5b919ba5900449358b3277530a7eff6c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:44:30.315755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed9faa-0f48-48f2-b1d3-31c65ca60a6d None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:30.316370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed9faa-0f48-48f2-b1d3-31c65ca60a6d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:30.316370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed9faa-0f48-48f2-b1d3-31c65ca60a6d None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:30.317682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 597/1233] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:44:30.323094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337277-0836-464a-846a-004373625b3d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:30.323515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337277-0836-464a-846a-004373625b3d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:30.323626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70337277-0836-464a-846a-004373625b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:30.342611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-70337277-0836-464a-846a-004373625b3d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:30.343800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-70337277-0836-464a-846a-004373625b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:30.409455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 637/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:44:30.677523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed9faa-0f48-48f2-b1d3-31c65ca60a6d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:30.704564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:30.788868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed9faa-0f48-48f2-b1d3-31c65ca60a6d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:30.824214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4d3ca-6a5d-45fb-8ed6-4a5978abeaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:30.824832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4d3ca-6a5d-45fb-8ed6-4a5978abeaad None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:30.825107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4d3ca-6a5d-45fb-8ed6-4a5978abeaad None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:30.825375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4d3ca-6a5d-45fb-8ed6-4a5978abeaad None admin] PATH_INFO: `/v3/roles/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:30.835134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4d3ca-6a5d-45fb-8ed6-4a5978abeaad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-31b4d3ca-6a5d-45fb-8ed6-4a5978abeaad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:44:30.839856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:30.841049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:30.841987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 598/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Oct 4 11:44:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:30.847360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70337277-0836-464a-846a-004373625b3d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:30.893977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:30.944600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70337277-0836-464a-846a-004373625b3d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:30.977366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98de99a-7d78-4763-a5b7-95b064431958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:30.977962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98de99a-7d78-4763-a5b7-95b064431958 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:30.978234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98de99a-7d78-4763-a5b7-95b064431958 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:30.978502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98de99a-7d78-4763-a5b7-95b064431958 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:30.979157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98de99a-7d78-4763-a5b7-95b064431958 None admin] RBAC: Adding query filter params (name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:30.979452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98de99a-7d78-4763-a5b7-95b064431958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:30.980833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98de99a-7d78-4763-a5b7-95b064431958 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:30.989471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 638/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Oct 4 11:44:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:30.995348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4d3ca-6a5d-45fb-8ed6-4a5978abeaad None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:31.079781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:31.095538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4d3ca-6a5d-45fb-8ed6-4a5978abeaad None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:31.131500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfa91b-2b70-49a2-a413-a5dea641c3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:31.132123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfa91b-2b70-49a2-a413-a5dea641c3df None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:31.132437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfa91b-2b70-49a2-a413-a5dea641c3df None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:31.132712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfa91b-2b70-49a2-a413-a5dea641c3df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:31.143244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfa91b-2b70-49a2-a413-a5dea641c3df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:31.144743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfa91b-2b70-49a2-a413-a5dea641c3df None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-0edfa91b-2b70-49a2-a413-a5dea641c3df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:31.160205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:31.161337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:31.162414 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:31.162414 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:31.162414 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:31.162414 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 599/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:31.167435 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98de99a-7d78-4763-a5b7-95b064431958 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:31.265248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98de99a-7d78-4763-a5b7-95b064431958 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:31.273908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:31.298834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de11dd-18dd-4159-b1fd-ab6b9ab8a31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:31.299431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de11dd-18dd-4159-b1fd-ab6b9ab8a31a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:31.299707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de11dd-18dd-4159-b1fd-ab6b9ab8a31a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:31.299973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de11dd-18dd-4159-b1fd-ab6b9ab8a31a None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:31.300632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de11dd-18dd-4159-b1fd-ab6b9ab8a31a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:31.300940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de11dd-18dd-4159-b1fd-ab6b9ab8a31a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:31.302294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de11dd-18dd-4159-b1fd-ab6b9ab8a31a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:31.313855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 639/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:31.319633 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfa91b-2b70-49a2-a413-a5dea641c3df None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:31.421177 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfa91b-2b70-49a2-a413-a5dea641c3df None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:31.458510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395e8ad-572c-4959-9509-108159df2d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:31.459134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395e8ad-572c-4959-9509-108159df2d51 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:31.459457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395e8ad-572c-4959-9509-108159df2d51 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:31.459817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395e8ad-572c-4959-9509-108159df2d51 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:31.462948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:31.468993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395e8ad-572c-4959-9509-108159df2d51 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:31.471052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395e8ad-572c-4959-9509-108159df2d51 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-9395e8ad-572c-4959-9509-108159df2d51 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:44:31.484071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:31.485247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:31.486290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 600/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:31.492010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de11dd-18dd-4159-b1fd-ab6b9ab8a31a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:31.603785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de11dd-18dd-4159-b1fd-ab6b9ab8a31a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:31.636573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87e3ed-3e8c-4f05-9504-13389fa9a1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:31.637165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87e3ed-3e8c-4f05-9504-13389fa9a1ca None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:31.637440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87e3ed-3e8c-4f05-9504-13389fa9a1ca None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:31.637709 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87e3ed-3e8c-4f05-9504-13389fa9a1ca None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:31.638420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87e3ed-3e8c-4f05-9504-13389fa9a1ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:31.638807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87e3ed-3e8c-4f05-9504-13389fa9a1ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:31.640132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87e3ed-3e8c-4f05-9504-13389fa9a1ca None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:31.658894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 640/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 11:44:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:31.664855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395e8ad-572c-4959-9509-108159df2d51 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:31.770340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395e8ad-572c-4959-9509-108159df2d51 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:31.805191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729c881-0fab-45a3-86c5-1cf3a35df9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:31.805800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729c881-0fab-45a3-86c5-1cf3a35df9b6 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:31.806080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729c881-0fab-45a3-86c5-1cf3a35df9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:31.806356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729c881-0fab-45a3-86c5-1cf3a35df9b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:31.869188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:31.876852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729c881-0fab-45a3-86c5-1cf3a35df9b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:31.878313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729c881-0fab-45a3-86c5-1cf3a35df9b6 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:31.974920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 601/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:44:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:44:31.980847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87e3ed-3e8c-4f05-9504-13389fa9a1ca None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:32.008422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:32.078790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87e3ed-3e8c-4f05-9504-13389fa9a1ca None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:32.112436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb093e-cfb7-4bb6-91ec-dc141183cb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:32.113065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb093e-cfb7-4bb6-91ec-dc141183cb14 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:32.113346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb093e-cfb7-4bb6-91ec-dc141183cb14 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:32.113621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb093e-cfb7-4bb6-91ec-dc141183cb14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:32.123371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb093e-cfb7-4bb6-91ec-dc141183cb14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:32.124791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb093e-cfb7-4bb6-91ec-dc141183cb14 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-25fb093e-cfb7-4bb6-91ec-dc141183cb14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:32.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:32.140961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:32.142142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:32.142142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:32.142142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:32.142142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 641/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:32.146609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729c881-0fab-45a3-86c5-1cf3a35df9b6 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:32.241870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:32.247908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729c881-0fab-45a3-86c5-1cf3a35df9b6 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:32.286146 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dd667-ed7e-48ab-988e-3768c508aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:32.286754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dd667-ed7e-48ab-988e-3768c508aa7c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:32.287031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dd667-ed7e-48ab-988e-3768c508aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:32.287304 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dd667-ed7e-48ab-988e-3768c508aa7c None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:32.287953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dd667-ed7e-48ab-988e-3768c508aa7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:32.288284 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dd667-ed7e-48ab-988e-3768c508aa7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:32.289679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dd667-ed7e-48ab-988e-3768c508aa7c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:32.301765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 602/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:44:32.307494 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb093e-cfb7-4bb6-91ec-dc141183cb14 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:32.388323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:32.409183 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb093e-cfb7-4bb6-91ec-dc141183cb14 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:32.442190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c86229-5051-4624-aae2-5619a0bfe7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:32.442823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c86229-5051-4624-aae2-5619a0bfe7bb None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:32.443322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c86229-5051-4624-aae2-5619a0bfe7bb None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:32.443826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c86229-5051-4624-aae2-5619a0bfe7bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:32.451019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c86229-5051-4624-aae2-5619a0bfe7bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:32.452488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c86229-5051-4624-aae2-5619a0bfe7bb None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-61c86229-5051-4624-aae2-5619a0bfe7bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:44:32.463785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:32.464987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:32.466312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:44:32.466312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:44:32.466312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:44:32.466312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:44:32.466312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:44:32.466312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:44:32.466312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:32.466312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:32.466312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 642/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:44:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:32.471538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dd667-ed7e-48ab-988e-3768c508aa7c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:32.576191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dd667-ed7e-48ab-988e-3768c508aa7c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:32.616560 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8fd13-4edf-420a-b6d3-5dd19ed2ae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:32.617170 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8fd13-4edf-420a-b6d3-5dd19ed2ae81 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:32.617451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8fd13-4edf-420a-b6d3-5dd19ed2ae81 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:32.617790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8fd13-4edf-420a-b6d3-5dd19ed2ae81 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:32.618485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8fd13-4edf-420a-b6d3-5dd19ed2ae81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:32.618787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8fd13-4edf-420a-b6d3-5dd19ed2ae81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:32.620286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8fd13-4edf-420a-b6d3-5dd19ed2ae81 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:32.632882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 603/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:44:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:32.639498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c86229-5051-4624-aae2-5619a0bfe7bb None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:32.742830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c86229-5051-4624-aae2-5619a0bfe7bb None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:32.776316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d3360-9b80-4daf-bf56-be1cb9cab1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:32.776967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d3360-9b80-4daf-bf56-be1cb9cab1fa None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:32.777248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d3360-9b80-4daf-bf56-be1cb9cab1fa None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:32.777519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d3360-9b80-4daf-bf56-be1cb9cab1fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:32.778235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d3360-9b80-4daf-bf56-be1cb9cab1fa None admin] RBAC: Adding query filter params (role.id=5b919ba5900449358b3277530a7eff6c, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=16374dbb6d084ddfa3b49afd19c7c325) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:32.778546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d3360-9b80-4daf-bf56-be1cb9cab1fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:32.779973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d3360-9b80-4daf-bf56-be1cb9cab1fa None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:32.788463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 643/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:44:32 2022] GET /identity/v3/role_assignments?user.id=16374dbb6d084ddfa3b49afd19c7c325&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=5b919ba5900449358b3277530a7eff6c => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:35.011551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009247-606b-4688-8a31-6e841beac8bd None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:35.012204 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009247-606b-4688-8a31-6e841beac8bd None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:35.012204 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009247-606b-4688-8a31-6e841beac8bd None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:35.013526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 604/1247] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:44:35.019716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579102b-ed16-4633-8725-e5d90df4ce54 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:35.020224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579102b-ed16-4633-8725-e5d90df4ce54 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:35.020328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579102b-ed16-4633-8725-e5d90df4ce54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:35.026609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:35.038743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d579102b-ed16-4633-8725-e5d90df4ce54 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:35.039943 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d579102b-ed16-4633-8725-e5d90df4ce54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:35.104765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 644/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:44:35.380632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07009247-606b-4688-8a31-6e841beac8bd None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:35.399427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:35.490126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07009247-606b-4688-8a31-6e841beac8bd None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:35.528296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b26607-ebcd-4755-b3ae-2cf3bd7ce40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:35.528924 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b26607-ebcd-4755-b3ae-2cf3bd7ce40c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:35.529198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b26607-ebcd-4755-b3ae-2cf3bd7ce40c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:35.529479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b26607-ebcd-4755-b3ae-2cf3bd7ce40c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:35.539831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b26607-ebcd-4755-b3ae-2cf3bd7ce40c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-94b26607-ebcd-4755-b3ae-2cf3bd7ce40c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:44:35.544815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:35.549240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:35.550388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 605/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:44:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:35.552623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579102b-ed16-4633-8725-e5d90df4ce54 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:35.650997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579102b-ed16-4633-8725-e5d90df4ce54 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:35.679665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:35.684646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15797e-54ee-4dd5-96de-32da0d6fe7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:35.685259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15797e-54ee-4dd5-96de-32da0d6fe7bf None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:35.685540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15797e-54ee-4dd5-96de-32da0d6fe7bf None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:35.685818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15797e-54ee-4dd5-96de-32da0d6fe7bf None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:35.686482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15797e-54ee-4dd5-96de-32da0d6fe7bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:35.686791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15797e-54ee-4dd5-96de-32da0d6fe7bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:35.688178 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15797e-54ee-4dd5-96de-32da0d6fe7bf None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:35.700067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 645/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:44:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:35.706745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b26607-ebcd-4755-b3ae-2cf3bd7ce40c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:35.784863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:35.808848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b26607-ebcd-4755-b3ae-2cf3bd7ce40c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:35.843446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18039287-c73f-411e-946a-33fcd2c308ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:35.844059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18039287-c73f-411e-946a-33fcd2c308ab None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:35.844374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18039287-c73f-411e-946a-33fcd2c308ab None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:35.844651 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18039287-c73f-411e-946a-33fcd2c308ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:35.854440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18039287-c73f-411e-946a-33fcd2c308ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:35.855882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18039287-c73f-411e-946a-33fcd2c308ab None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-18039287-c73f-411e-946a-33fcd2c308ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:35.870869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:35.872215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:35.873458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:35.873458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:35.873458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:35.873458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 606/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:35.878506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15797e-54ee-4dd5-96de-32da0d6fe7bf None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:35.978221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15797e-54ee-4dd5-96de-32da0d6fe7bf None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:36.011909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838088d-825e-4f1c-a461-72ace1da8dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:36.012531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838088d-825e-4f1c-a461-72ace1da8dd7 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:36.012812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838088d-825e-4f1c-a461-72ace1da8dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:36.013084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838088d-825e-4f1c-a461-72ace1da8dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:36.013715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838088d-825e-4f1c-a461-72ace1da8dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:36.014018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838088d-825e-4f1c-a461-72ace1da8dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:36.015516 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838088d-825e-4f1c-a461-72ace1da8dd7 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:36.027217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 646/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:36.033049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18039287-c73f-411e-946a-33fcd2c308ab None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:36.134256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18039287-c73f-411e-946a-33fcd2c308ab None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:36.160010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:36.167673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4070c9-b604-4d9f-b577-9aa04b676b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:36.169851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4070c9-b604-4d9f-b577-9aa04b676b67 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:36.169851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4070c9-b604-4d9f-b577-9aa04b676b67 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:36.169851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4070c9-b604-4d9f-b577-9aa04b676b67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:36.177169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4070c9-b604-4d9f-b577-9aa04b676b67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:36.178607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4070c9-b604-4d9f-b577-9aa04b676b67 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-6e4070c9-b604-4d9f-b577-9aa04b676b67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:44:36.190477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:36.197207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:36.198419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 607/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:36.199140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838088d-825e-4f1c-a461-72ace1da8dd7 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:36.253026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:36.295438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838088d-825e-4f1c-a461-72ace1da8dd7 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:36.329184 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066af64b-6482-497d-83cf-ac1c1839ffd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:36.329779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066af64b-6482-497d-83cf-ac1c1839ffd1 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:36.330053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066af64b-6482-497d-83cf-ac1c1839ffd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:36.330321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066af64b-6482-497d-83cf-ac1c1839ffd1 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:36.331033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066af64b-6482-497d-83cf-ac1c1839ffd1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:36.331328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066af64b-6482-497d-83cf-ac1c1839ffd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:36.332792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066af64b-6482-497d-83cf-ac1c1839ffd1 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:36.349923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 647/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 11:44:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:36.355871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4070c9-b604-4d9f-b577-9aa04b676b67 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:36.458327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4070c9-b604-4d9f-b577-9aa04b676b67 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:36.493716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b1d40-1b40-4a45-acb2-1e6c1a02c715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:36.494345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b1d40-1b40-4a45-acb2-1e6c1a02c715 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:36.494626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b1d40-1b40-4a45-acb2-1e6c1a02c715 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:36.494899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b1d40-1b40-4a45-acb2-1e6c1a02c715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:36.554054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:36.566343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b1d40-1b40-4a45-acb2-1e6c1a02c715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:36.567829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b1d40-1b40-4a45-acb2-1e6c1a02c715 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:36.665082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 608/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:44:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:44:36.671242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066af64b-6482-497d-83cf-ac1c1839ffd1 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:36.772774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066af64b-6482-497d-83cf-ac1c1839ffd1 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:36.786619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:36.806942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310c031-d3de-47c9-a000-3d0cc3316b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:36.807530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310c031-d3de-47c9-a000-3d0cc3316b4a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:36.807809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310c031-d3de-47c9-a000-3d0cc3316b4a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:36.808157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310c031-d3de-47c9-a000-3d0cc3316b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:36.818515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310c031-d3de-47c9-a000-3d0cc3316b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:36.823546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310c031-d3de-47c9-a000-3d0cc3316b4a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-6310c031-d3de-47c9-a000-3d0cc3316b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:36.835027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:36.836230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:36.837377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:36.837377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:36.837377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:36.837377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 648/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:36.842026 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b1d40-1b40-4a45-acb2-1e6c1a02c715 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:36.931281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:36.944457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b1d40-1b40-4a45-acb2-1e6c1a02c715 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:36.980463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2951cb-12ec-4539-b69a-69113ebac22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:36.981065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2951cb-12ec-4539-b69a-69113ebac22d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:36.981337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2951cb-12ec-4539-b69a-69113ebac22d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:36.981626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2951cb-12ec-4539-b69a-69113ebac22d None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:36.982256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2951cb-12ec-4539-b69a-69113ebac22d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:36.982556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2951cb-12ec-4539-b69a-69113ebac22d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:36.983905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2951cb-12ec-4539-b69a-69113ebac22d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:36.996049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 609/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:37.002430 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310c031-d3de-47c9-a000-3d0cc3316b4a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:37.102157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310c031-d3de-47c9-a000-3d0cc3316b4a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:37.136408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e11c6a-8c1c-4806-a1e5-5d8ca431a648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:37.137040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e11c6a-8c1c-4806-a1e5-5d8ca431a648 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:37.137325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e11c6a-8c1c-4806-a1e5-5d8ca431a648 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:37.137602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e11c6a-8c1c-4806-a1e5-5d8ca431a648 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:37.145800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e11c6a-8c1c-4806-a1e5-5d8ca431a648 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:37.147236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e11c6a-8c1c-4806-a1e5-5d8ca431a648 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-94e11c6a-8c1c-4806-a1e5-5d8ca431a648 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:44:37.158499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:37.159609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:37.160973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:44:37.160973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:44:37.160973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:44:37.160973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:44:37.160973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:44:37.160973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:44:37.160973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:37.160973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:37.160973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 649/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:44:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:37.165905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2951cb-12ec-4539-b69a-69113ebac22d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:37.276806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2951cb-12ec-4539-b69a-69113ebac22d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:37.315860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2796f-38ae-4b4d-a222-e4bec3537307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:37.316494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2796f-38ae-4b4d-a222-e4bec3537307 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:37.316775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2796f-38ae-4b4d-a222-e4bec3537307 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:37.317046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2796f-38ae-4b4d-a222-e4bec3537307 None admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:37.317726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2796f-38ae-4b4d-a222-e4bec3537307 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:37.318031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2796f-38ae-4b4d-a222-e4bec3537307 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:37.319455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2796f-38ae-4b4d-a222-e4bec3537307 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:37.331541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 610/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:44:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:37.337792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e11c6a-8c1c-4806-a1e5-5d8ca431a648 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:37.364220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:37.441554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e11c6a-8c1c-4806-a1e5-5d8ca431a648 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:37.473020 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae382f6d-acdb-4bf7-acde-1b7c7500aabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:37.473616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae382f6d-acdb-4bf7-acde-1b7c7500aabf None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:37.473892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae382f6d-acdb-4bf7-acde-1b7c7500aabf None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:37.474162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae382f6d-acdb-4bf7-acde-1b7c7500aabf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:37.474865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae382f6d-acdb-4bf7-acde-1b7c7500aabf None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=16374dbb6d084ddfa3b49afd19c7c325) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:37.475166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae382f6d-acdb-4bf7-acde-1b7c7500aabf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:37.476605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae382f6d-acdb-4bf7-acde-1b7c7500aabf None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:37.481822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 650/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:44:37 2022] GET /identity/v3/role_assignments?user.id=16374dbb6d084ddfa3b49afd19c7c325&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:39.649972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c7ab-a8b0-4e1f-88d2-a96809846f15 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:39.650711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c7ab-a8b0-4e1f-88d2-a96809846f15 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:39.650711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c7ab-a8b0-4e1f-88d2-a96809846f15 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:39.651869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 611/1261] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:44:39.658444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddec7f-e6e7-455f-b528-9dd5592e49c2 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:39.658971 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddec7f-e6e7-455f-b528-9dd5592e49c2 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:39.659078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddec7f-e6e7-455f-b528-9dd5592e49c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:39.677566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f3ddec7f-e6e7-455f-b528-9dd5592e49c2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:39.678765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f3ddec7f-e6e7-455f-b528-9dd5592e49c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:39.749720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 651/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:44:40.027209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c7ab-a8b0-4e1f-88d2-a96809846f15 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:40.038372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:40.129012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c7ab-a8b0-4e1f-88d2-a96809846f15 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:40.164390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938a9ae-428c-42b2-9460-a2fb51469dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:40.165012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938a9ae-428c-42b2-9460-a2fb51469dcf None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:40.165291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938a9ae-428c-42b2-9460-a2fb51469dcf None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:40.165562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938a9ae-428c-42b2-9460-a2fb51469dcf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:40.175804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938a9ae-428c-42b2-9460-a2fb51469dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-8938a9ae-428c-42b2-9460-a2fb51469dcf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:44:40.180775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:40.186908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:40.188112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 612/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:44:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:40.189712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddec7f-e6e7-455f-b528-9dd5592e49c2 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:40.208098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:40.288423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddec7f-e6e7-455f-b528-9dd5592e49c2 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:40.322638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10140d-1525-4ed6-aed8-8e3b4b31e290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:40.323245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10140d-1525-4ed6-aed8-8e3b4b31e290 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:40.323529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10140d-1525-4ed6-aed8-8e3b4b31e290 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:40.323805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10140d-1525-4ed6-aed8-8e3b4b31e290 None admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:40.324499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10140d-1525-4ed6-aed8-8e3b4b31e290 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:40.324856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10140d-1525-4ed6-aed8-8e3b4b31e290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:40.326214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10140d-1525-4ed6-aed8-8e3b4b31e290 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:40.335110 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 652/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:44:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:44:40.341182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938a9ae-428c-42b2-9460-a2fb51469dcf None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:40.408956 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:40.442698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938a9ae-428c-42b2-9460-a2fb51469dcf None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:40.476662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338a6f1-afc0-45d3-9b60-973883ec5d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:40.477308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338a6f1-afc0-45d3-9b60-973883ec5d89 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:40.477603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338a6f1-afc0-45d3-9b60-973883ec5d89 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:40.477870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338a6f1-afc0-45d3-9b60-973883ec5d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:40.489297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338a6f1-afc0-45d3-9b60-973883ec5d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:40.494306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338a6f1-afc0-45d3-9b60-973883ec5d89 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-6338a6f1-afc0-45d3-9b60-973883ec5d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:40.506001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:40.507190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:40.508307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:40.508307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:40.508307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:40.508307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 613/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:40.513220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10140d-1525-4ed6-aed8-8e3b4b31e290 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:40.599680 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:40.620602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10140d-1525-4ed6-aed8-8e3b4b31e290 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:40.654876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350a2d1-e603-4e37-9924-48274b858ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:40.655480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350a2d1-e603-4e37-9924-48274b858ebb None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:40.655762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350a2d1-e603-4e37-9924-48274b858ebb None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:40.656037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350a2d1-e603-4e37-9924-48274b858ebb None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:40.656704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350a2d1-e603-4e37-9924-48274b858ebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:40.657009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350a2d1-e603-4e37-9924-48274b858ebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:40.658552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350a2d1-e603-4e37-9924-48274b858ebb None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:40.670717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 653/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:40.677102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338a6f1-afc0-45d3-9b60-973883ec5d89 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:40.788427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338a6f1-afc0-45d3-9b60-973883ec5d89 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:40.809698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:40.823153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c1663-600d-451f-8e5a-7fa1c30e7760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:40.823772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c1663-600d-451f-8e5a-7fa1c30e7760 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:40.824053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c1663-600d-451f-8e5a-7fa1c30e7760 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:40.824433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c1663-600d-451f-8e5a-7fa1c30e7760 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:40.834337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c1663-600d-451f-8e5a-7fa1c30e7760 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:40.835778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c1663-600d-451f-8e5a-7fa1c30e7760 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-fd7c1663-600d-451f-8e5a-7fa1c30e7760 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:44:40.847432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:40.848596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:40.849590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 614/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 11:44:40.854466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350a2d1-e603-4e37-9924-48274b858ebb None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:40.952835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350a2d1-e603-4e37-9924-48274b858ebb None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:40.988432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca4b6d-df06-41dc-9d59-17aff7f46973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:40.989025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca4b6d-df06-41dc-9d59-17aff7f46973 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:40.989300 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca4b6d-df06-41dc-9d59-17aff7f46973 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:40.989569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca4b6d-df06-41dc-9d59-17aff7f46973 None admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:40.990283 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca4b6d-df06-41dc-9d59-17aff7f46973 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:40.990585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca4b6d-df06-41dc-9d59-17aff7f46973 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:40.992069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca4b6d-df06-41dc-9d59-17aff7f46973 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:41.000552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:41.008950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 654/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 11:44:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:41.014807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c1663-600d-451f-8e5a-7fa1c30e7760 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:41.118702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c1663-600d-451f-8e5a-7fa1c30e7760 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:41.152147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9915a-65c4-433a-b8e9-22393b979f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:41.152790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9915a-65c4-433a-b8e9-22393b979f3b None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:41.153068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9915a-65c4-433a-b8e9-22393b979f3b None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:41.153340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9915a-65c4-433a-b8e9-22393b979f3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:41.164427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9915a-65c4-433a-b8e9-22393b979f3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:41.165863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9915a-65c4-433a-b8e9-22393b979f3b None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-7cb9915a-65c4-433a-b8e9-22393b979f3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:41.179606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:41.185604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:41.186782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:41.186782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:41.186782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:41.186782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 615/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:41.187073 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca4b6d-df06-41dc-9d59-17aff7f46973 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:41.289258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca4b6d-df06-41dc-9d59-17aff7f46973 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:41.322134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bf443-2d78-4451-87eb-475463a2cd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:41.322738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bf443-2d78-4451-87eb-475463a2cd5a None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:41.323013 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bf443-2d78-4451-87eb-475463a2cd5a None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:41.323281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bf443-2d78-4451-87eb-475463a2cd5a None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:41.323906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bf443-2d78-4451-87eb-475463a2cd5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:41.324237 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bf443-2d78-4451-87eb-475463a2cd5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:41.325605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bf443-2d78-4451-87eb-475463a2cd5a None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:41.337258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 655/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:41.342897 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9915a-65c4-433a-b8e9-22393b979f3b None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:41.374643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:41.442574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9915a-65c4-433a-b8e9-22393b979f3b None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:41.477141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d58297-25c5-4e8e-b081-bd6d32777f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:41.477762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d58297-25c5-4e8e-b081-bd6d32777f7d None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:41.478046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d58297-25c5-4e8e-b081-bd6d32777f7d None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:41.478323 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d58297-25c5-4e8e-b081-bd6d32777f7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:41.485126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d58297-25c5-4e8e-b081-bd6d32777f7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:41.486532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d58297-25c5-4e8e-b081-bd6d32777f7d None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-d6d58297-25c5-4e8e-b081-bd6d32777f7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:44:41.497764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:41.498869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:41.499959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:44:41.499959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:44:41.499959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:44:41.499959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:44:41.499959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:44:41.499959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:44:41.499959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:41.499959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:41.499959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 616/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:44:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:41.505379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bf443-2d78-4451-87eb-475463a2cd5a None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:41.562876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:41.604342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bf443-2d78-4451-87eb-475463a2cd5a None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:41.637350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dded925-06c3-457a-9b0b-e770ebcf02ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:41.637552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dded925-06c3-457a-9b0b-e770ebcf02ff None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:41.637825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dded925-06c3-457a-9b0b-e770ebcf02ff None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:41.638091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dded925-06c3-457a-9b0b-e770ebcf02ff None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:41.638761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dded925-06c3-457a-9b0b-e770ebcf02ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:41.639058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dded925-06c3-457a-9b0b-e770ebcf02ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:41.640496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dded925-06c3-457a-9b0b-e770ebcf02ff None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:41.652686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 656/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:44:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:44:41.658768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d58297-25c5-4e8e-b081-bd6d32777f7d None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:41.752700 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:41.761115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d58297-25c5-4e8e-b081-bd6d32777f7d None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:41.794853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cff207-c27e-4958-974d-caf882f458b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:41.795609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cff207-c27e-4958-974d-caf882f458b1 None admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:41.795965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cff207-c27e-4958-974d-caf882f458b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:41.796357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cff207-c27e-4958-974d-caf882f458b1 None admin] PATH_INFO: `/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/16374dbb6d084ddfa3b49afd19c7c325/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:41.798844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cff207-c27e-4958-974d-caf882f458b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=db48c0e368b84afdba6bdfe3c37065f0, user_id=16374dbb6d084ddfa3b49afd19c7c325, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:41.800321 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cff207-c27e-4958-974d-caf882f458b1 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:41.908249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 617/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Oct 4 11:44:41 2022] PUT /identity/v3/projects/db48c0e368b84afdba6bdfe3c37065f0/users/16374dbb6d084ddfa3b49afd19c7c325/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 11:44:44.105839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f68907-fcd2-4a15-a751-c7fd86b5bfbc None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:44.106392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f68907-fcd2-4a15-a751-c7fd86b5bfbc None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:44.106392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f68907-fcd2-4a15-a751-c7fd86b5bfbc None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:44.107729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 657/1274] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:44:44.113806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293128-8ba3-42e0-a9f5-c9f8bda42369 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:44.114664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293128-8ba3-42e0-a9f5-c9f8bda42369 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:44.114664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293128-8ba3-42e0-a9f5-c9f8bda42369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:44.133620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-cc293128-8ba3-42e0-a9f5-c9f8bda42369 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:44.134907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cc293128-8ba3-42e0-a9f5-c9f8bda42369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:44.205967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 618/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:44:44.488380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f68907-fcd2-4a15-a751-c7fd86b5bfbc None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:44.603587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f68907-fcd2-4a15-a751-c7fd86b5bfbc None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:44.628325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:44.655231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8b81c-ecda-4877-ba1d-656e91ecd7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:44.656330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8b81c-ecda-4877-ba1d-656e91ecd7cc None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:44.656825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8b81c-ecda-4877-ba1d-656e91ecd7cc None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:44.657827 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8b81c-ecda-4877-ba1d-656e91ecd7cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:44.675594 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8b81c-ecda-4877-ba1d-656e91ecd7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-93f8b81c-ecda-4877-ba1d-656e91ecd7cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 11:44:44.685392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:44.686431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 11:44:44.687295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:44.687961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 658/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Oct 4 11:44:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:44.693614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293128-8ba3-42e0-a9f5-c9f8bda42369 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:44.776682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:44.795826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293128-8ba3-42e0-a9f5-c9f8bda42369 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:44.833561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f12757-bdab-40c9-9017-c410236c80df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:44.834193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f12757-bdab-40c9-9017-c410236c80df None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:44.834474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f12757-bdab-40c9-9017-c410236c80df None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:44.834752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f12757-bdab-40c9-9017-c410236c80df None admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:44.835415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f12757-bdab-40c9-9017-c410236c80df None admin] RBAC: Adding query filter params (name=admin) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:44.835759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f12757-bdab-40c9-9017-c410236c80df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:44.837162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f12757-bdab-40c9-9017-c410236c80df None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:44.846071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 619/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Oct 4 11:44:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:44.852536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8b81c-ecda-4877-ba1d-656e91ecd7cc None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:44.951483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8b81c-ecda-4877-ba1d-656e91ecd7cc None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:44.986579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65074fcc-620b-47ad-8e3a-ec6aa1c6cee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:44.987205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65074fcc-620b-47ad-8e3a-ec6aa1c6cee3 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:44.987483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65074fcc-620b-47ad-8e3a-ec6aa1c6cee3 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:44.987757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65074fcc-620b-47ad-8e3a-ec6aa1c6cee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:44.997847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65074fcc-620b-47ad-8e3a-ec6aa1c6cee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:44.999278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65074fcc-620b-47ad-8e3a-ec6aa1c6cee3 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-65074fcc-620b-47ad-8e3a-ec6aa1c6cee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:45.014091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:45.015469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:45.017380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:45.017380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:45.017380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:45.017380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 659/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:45.022001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f12757-bdab-40c9-9017-c410236c80df None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:45.126203 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f12757-bdab-40c9-9017-c410236c80df None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:45.160121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551684b-7467-47f8-b8f6-6fd45437ca9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:45.160755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551684b-7467-47f8-b8f6-6fd45437ca9c None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:45.161034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551684b-7467-47f8-b8f6-6fd45437ca9c None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:45.161309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551684b-7467-47f8-b8f6-6fd45437ca9c None admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:45.161950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551684b-7467-47f8-b8f6-6fd45437ca9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:45.162329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551684b-7467-47f8-b8f6-6fd45437ca9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:45.163637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551684b-7467-47f8-b8f6-6fd45437ca9c None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:45.175463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 620/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 11:44:45.181400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65074fcc-620b-47ad-8e3a-ec6aa1c6cee3 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:45.214067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:45.289063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65074fcc-620b-47ad-8e3a-ec6aa1c6cee3 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:45.322662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0be51e-ec92-4a4b-8d22-685ce5188879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:45.323292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0be51e-ec92-4a4b-8d22-685ce5188879 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:45.323577 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0be51e-ec92-4a4b-8d22-685ce5188879 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:45.323853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0be51e-ec92-4a4b-8d22-685ce5188879 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:45.332273 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0be51e-ec92-4a4b-8d22-685ce5188879 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:45.333718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0be51e-ec92-4a4b-8d22-685ce5188879 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-cb0be51e-ec92-4a4b-8d22-685ce5188879 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 11:44:45.345527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:45.346600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:45.347698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:45.347698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:45.347698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 11:44:45.347698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 11:44:45.347698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 11:44:45.347698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 11:44:45.347698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 11:44:45.347698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 11:44:45.347698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 11:44:45.347698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:45.350763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 660/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:45.352557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551684b-7467-47f8-b8f6-6fd45437ca9c None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:45.356534 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:45.451847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551684b-7467-47f8-b8f6-6fd45437ca9c None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:45.487101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8dcf7b-cc2a-4f2e-baab-617f5aa55048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:45.487696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dcf7b-cc2a-4f2e-baab-617f5aa55048 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:45.487977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dcf7b-cc2a-4f2e-baab-617f5aa55048 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:45.488291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8dcf7b-cc2a-4f2e-baab-617f5aa55048 None admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:45.489015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8dcf7b-cc2a-4f2e-baab-617f5aa55048 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:45.489320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8dcf7b-cc2a-4f2e-baab-617f5aa55048 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:45.490742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8dcf7b-cc2a-4f2e-baab-617f5aa55048 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:45.508756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 621/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Oct 4 11:44:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:45.514734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0be51e-ec92-4a4b-8d22-685ce5188879 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:45.606621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:45.616592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0be51e-ec92-4a4b-8d22-685ce5188879 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:45.649181 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3ae68-6f7d-44a7-9ab3-f040952bf076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:45.650046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3ae68-6f7d-44a7-9ab3-f040952bf076 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:45.650320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3ae68-6f7d-44a7-9ab3-f040952bf076 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:45.650588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3ae68-6f7d-44a7-9ab3-f040952bf076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:45.719270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3ae68-6f7d-44a7-9ab3-f040952bf076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:45.720735 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3ae68-6f7d-44a7-9ab3-f040952bf076 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:45.810101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:45.820655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 661/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Oct 4 11:44:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 11:44:45.826600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8dcf7b-cc2a-4f2e-baab-617f5aa55048 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:45.888943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:45.927822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8dcf7b-cc2a-4f2e-baab-617f5aa55048 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:45.962022 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3783a-4574-4ac6-8b1a-e5f6f1c669cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:45.962637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3783a-4574-4ac6-8b1a-e5f6f1c669cb None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:45.962919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3783a-4574-4ac6-8b1a-e5f6f1c669cb None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:45.963190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3783a-4574-4ac6-8b1a-e5f6f1c669cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:45.973462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3783a-4574-4ac6-8b1a-e5f6f1c669cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:45.974828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3783a-4574-4ac6-8b1a-e5f6f1c669cb None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-8fc3783a-4574-4ac6-8b1a-e5f6f1c669cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 11:44:45.989561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:45.990729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 11:44:45.991870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 11:44:45.991870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 11:44:45.991870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:45.991870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 622/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:45.996431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3ae68-6f7d-44a7-9ab3-f040952bf076 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:46.104797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3ae68-6f7d-44a7-9ab3-f040952bf076 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:46.139306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214459bb-066e-45f5-9c17-408d917632a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:46.139962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214459bb-066e-45f5-9c17-408d917632a6 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:46.140215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214459bb-066e-45f5-9c17-408d917632a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:46.140496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214459bb-066e-45f5-9c17-408d917632a6 None admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:46.141131 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214459bb-066e-45f5-9c17-408d917632a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:46.141439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214459bb-066e-45f5-9c17-408d917632a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:46.142837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214459bb-066e-45f5-9c17-408d917632a6 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:46.153827 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 662/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Oct 4 11:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:46.160071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3783a-4574-4ac6-8b1a-e5f6f1c669cb None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:46.259129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3783a-4574-4ac6-8b1a-e5f6f1c669cb None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:46.273155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:46.293720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25597149-8059-4fca-b295-48ad39614b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:46.294346 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25597149-8059-4fca-b295-48ad39614b54 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:46.294622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25597149-8059-4fca-b295-48ad39614b54 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:46.294894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25597149-8059-4fca-b295-48ad39614b54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:46.302102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25597149-8059-4fca-b295-48ad39614b54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:46.303528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25597149-8059-4fca-b295-48ad39614b54 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application [None req-25597149-8059-4fca-b295-48ad39614b54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 11:44:46.315692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:46.318999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:46.320240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 11:44:46.320240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 11:44:46.320240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 11:44:46.320240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 11:44:46.320240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 11:44:46.320240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 11:44:46.320240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 11:44:46.320240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: ERROR keystone.server.flask.application Oct 04 11:44:46.320240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 623/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Oct 4 11:44:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:46.322897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214459bb-066e-45f5-9c17-408d917632a6 None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:46.374320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:46.423868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214459bb-066e-45f5-9c17-408d917632a6 None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:46.457341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89794d-ad85-4a31-889c-45b699631ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:46.457943 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89794d-ad85-4a31-889c-45b699631ddc None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:46.458279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89794d-ad85-4a31-889c-45b699631ddc None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:46.458551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89794d-ad85-4a31-889c-45b699631ddc None admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:46.459228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89794d-ad85-4a31-889c-45b699631ddc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:46.459530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89794d-ad85-4a31-889c-45b699631ddc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:46.460960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89794d-ad85-4a31-889c-45b699631ddc None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:46.473228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 663/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Oct 4 11:44:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:46.479333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25597149-8059-4fca-b295-48ad39614b54 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:46.578747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25597149-8059-4fca-b295-48ad39614b54 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:46.613117 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bae6fb-3523-42fa-99b4-e1d5a9ecaa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:46.613712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bae6fb-3523-42fa-99b4-e1d5a9ecaa6f None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:46.613992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bae6fb-3523-42fa-99b4-e1d5a9ecaa6f None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:46.614259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bae6fb-3523-42fa-99b4-e1d5a9ecaa6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:46.614965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bae6fb-3523-42fa-99b4-e1d5a9ecaa6f None admin] RBAC: Adding query filter params (role.id=fce271509b2c4922bd9cf3f987c9e34d, scope.project.id=db48c0e368b84afdba6bdfe3c37065f0, user.id=16374dbb6d084ddfa3b49afd19c7c325) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:46.615271 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bae6fb-3523-42fa-99b4-e1d5a9ecaa6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:46.616729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bae6fb-3523-42fa-99b4-e1d5a9ecaa6f None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:46.622575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 624/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Oct 4 11:44:46 2022] GET /identity/v3/role_assignments?user.id=16374dbb6d084ddfa3b49afd19c7c325&scope.project.id=db48c0e368b84afdba6bdfe3c37065f0&role.id=fce271509b2c4922bd9cf3f987c9e34d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:48.847459 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbdd86-9561-46f6-ae85-9176d3cc25cd None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:48.848051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbdd86-9561-46f6-ae85-9176d3cc25cd None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:48.848051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbdd86-9561-46f6-ae85-9176d3cc25cd None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:48.849387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 664/1288] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:44:48.854557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcfb8b-d8d1-4fe2-b420-ddfe51fbca91 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:48.855011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcfb8b-d8d1-4fe2-b420-ddfe51fbca91 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:48.855145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcfb8b-d8d1-4fe2-b420-ddfe51fbca91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:48.874251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-8cfcfb8b-d8d1-4fe2-b420-ddfe51fbca91 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:48.875500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8cfcfb8b-d8d1-4fe2-b420-ddfe51fbca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:48.906843 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:48.944594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 625/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:44:49.215393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbdd86-9561-46f6-ae85-9176d3cc25cd None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:49.316165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbdd86-9561-46f6-ae85-9176d3cc25cd None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:49.327112 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:49.350855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0868d9c-6f1f-47dd-a804-89faffde177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:49.351550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868d9c-6f1f-47dd-a804-89faffde177d None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:49.351834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868d9c-6f1f-47dd-a804-89faffde177d None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:49.352128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0868d9c-6f1f-47dd-a804-89faffde177d None admin] PATH_INFO: `/v3/services/placement` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:49.352787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0868d9c-6f1f-47dd-a804-89faffde177d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:49.354164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0868d9c-6f1f-47dd-a804-89faffde177d None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-a0868d9c-6f1f-47dd-a804-89faffde177d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:49.363804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:49.365106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:49.366095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 665/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 11:44:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:49.371398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcfb8b-d8d1-4fe2-b420-ddfe51fbca91 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:49.469408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcfb8b-d8d1-4fe2-b420-ddfe51fbca91 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:49.507027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0e21c-3986-4188-b0ce-a1633bb63048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:49.507734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0e21c-3986-4188-b0ce-a1633bb63048 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:49.508060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0e21c-3986-4188-b0ce-a1633bb63048 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:49.508406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0e21c-3986-4188-b0ce-a1633bb63048 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:49.509079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0e21c-3986-4188-b0ce-a1633bb63048 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:49.509424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0e21c-3986-4188-b0ce-a1633bb63048 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:49.510858 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0e21c-3986-4188-b0ce-a1633bb63048 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:49.515782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 626/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 4 11:44:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:49.522137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0868d9c-6f1f-47dd-a804-89faffde177d None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:49.624861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0868d9c-6f1f-47dd-a804-89faffde177d None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:49.660651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21858a-20dc-487d-a7a9-c4efbb199749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:49.661337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21858a-20dc-487d-a7a9-c4efbb199749 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:49.661614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21858a-20dc-487d-a7a9-c4efbb199749 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:49.661881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21858a-20dc-487d-a7a9-c4efbb199749 None admin] PATH_INFO: `/v3/services` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:49.662500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21858a-20dc-487d-a7a9-c4efbb199749 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:49.662798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21858a-20dc-487d-a7a9-c4efbb199749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:49.664155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21858a-20dc-487d-a7a9-c4efbb199749 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:49.668572 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 666/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 4 11:44:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:51.881396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8c8c3-fcdd-43e4-9421-c278b2f8b9ee None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:51.881967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8c8c3-fcdd-43e4-9421-c278b2f8b9ee None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:51.881967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8c8c3-fcdd-43e4-9421-c278b2f8b9ee None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:51.883255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 627/1293] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:44:51.888462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122056a-ed36-4919-b488-9c399d4a9b44 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:51.888883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122056a-ed36-4919-b488-9c399d4a9b44 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:51.889015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122056a-ed36-4919-b488-9c399d4a9b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:51.907482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-0122056a-ed36-4919-b488-9c399d4a9b44 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:51.908602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-0122056a-ed36-4919-b488-9c399d4a9b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:51.939070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:51.976342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 667/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 11:44:52.248698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8c8c3-fcdd-43e4-9421-c278b2f8b9ee None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:52.279603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:52.351398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8c8c3-fcdd-43e4-9421-c278b2f8b9ee None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:52.384299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029c9bf-8f53-4884-bbd2-e4b9b3d3f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:52.384900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029c9bf-8f53-4884-bbd2-e4b9b3d3f764 None admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:52.385181 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029c9bf-8f53-4884-bbd2-e4b9b3d3f764 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:52.385456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029c9bf-8f53-4884-bbd2-e4b9b3d3f764 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:52.386111 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029c9bf-8f53-4884-bbd2-e4b9b3d3f764 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:52.387466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029c9bf-8f53-4884-bbd2-e4b9b3d3f764 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:52.496473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 628/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Oct 4 11:44:52 2022] POST /identity/v3/services => generated 246 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:44:54.706029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76a085-647a-4b5a-83cd-9dac5fbe3444 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:54.706629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76a085-647a-4b5a-83cd-9dac5fbe3444 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:54.706629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76a085-647a-4b5a-83cd-9dac5fbe3444 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:54.707975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 668/1296] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:44:54.714073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0442216-2695-4ab1-bfa0-4c6b59704c10 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:54.714629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0442216-2695-4ab1-bfa0-4c6b59704c10 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:54.714629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0442216-2695-4ab1-bfa0-4c6b59704c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:54.734713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d0442216-2695-4ab1-bfa0-4c6b59704c10 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:54.736288 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d0442216-2695-4ab1-bfa0-4c6b59704c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:54.819509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 629/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:44:55.097264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76a085-647a-4b5a-83cd-9dac5fbe3444 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:55.211187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76a085-647a-4b5a-83cd-9dac5fbe3444 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:55.251510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4787fa5-4c1e-485b-8773-e73bd32f9d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:55.252202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4787fa5-4c1e-485b-8773-e73bd32f9d9f None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:55.252472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4787fa5-4c1e-485b-8773-e73bd32f9d9f None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:55.252745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4787fa5-4c1e-485b-8773-e73bd32f9d9f None admin] PATH_INFO: `/v3/services/placement` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:55.253348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4787fa5-4c1e-485b-8773-e73bd32f9d9f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:55.254969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4787fa5-4c1e-485b-8773-e73bd32f9d9f None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-e4787fa5-4c1e-485b-8773-e73bd32f9d9f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:55.264914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:55.266668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:55.267569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 669/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 11:44:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:55.272977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0442216-2695-4ab1-bfa0-4c6b59704c10 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:55.294727 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:55.404744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0442216-2695-4ab1-bfa0-4c6b59704c10 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:55.452557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34596059-c1dd-4c1f-ad09-cc042d350a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:55.453168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34596059-c1dd-4c1f-ad09-cc042d350a02 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:55.453448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34596059-c1dd-4c1f-ad09-cc042d350a02 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:55.453730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34596059-c1dd-4c1f-ad09-cc042d350a02 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:55.455014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34596059-c1dd-4c1f-ad09-cc042d350a02 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:55.455314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34596059-c1dd-4c1f-ad09-cc042d350a02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:55.456761 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34596059-c1dd-4c1f-ad09-cc042d350a02 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:55.462574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 630/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 4 11:44:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:55.468891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4787fa5-4c1e-485b-8773-e73bd32f9d9f None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:55.503487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:55.572682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4787fa5-4c1e-485b-8773-e73bd32f9d9f None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:55.606963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76699cd-8740-4e41-9edf-2b5e7ff08943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:55.607568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76699cd-8740-4e41-9edf-2b5e7ff08943 None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:55.607901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76699cd-8740-4e41-9edf-2b5e7ff08943 None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:55.608196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76699cd-8740-4e41-9edf-2b5e7ff08943 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:55.608940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76699cd-8740-4e41-9edf-2b5e7ff08943 None admin] RBAC: Adding query filter params (interface=public, service_id=b4d95c266b8f48d39418763eb1574cd0, region_id=RegionOne) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:55.609245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76699cd-8740-4e41-9edf-2b5e7ff08943 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:55.610886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76699cd-8740-4e41-9edf-2b5e7ff08943 None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:55.615978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 670/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Oct 4 11:44:55 2022] GET /identity/v3/endpoints?service_id=b4d95c266b8f48d39418763eb1574cd0&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:55.621907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34596059-c1dd-4c1f-ad09-cc042d350a02 None admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:55.708494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:55.726698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34596059-c1dd-4c1f-ad09-cc042d350a02 None admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:55.760189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2c509-bed9-48f0-a2db-b04f30d4b216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:55.760828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2c509-bed9-48f0-a2db-b04f30d4b216 None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:55.761131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2c509-bed9-48f0-a2db-b04f30d4b216 None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:55.761401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2c509-bed9-48f0-a2db-b04f30d4b216 None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:55.762033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2c509-bed9-48f0-a2db-b04f30d4b216 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:55.763498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2c509-bed9-48f0-a2db-b04f30d4b216 None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:55.770991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 631/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Oct 4 11:44:55 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 11:44:57.912137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c6deb-f057-4573-ba0c-8bce11328958 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:57.912716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c6deb-f057-4573-ba0c-8bce11328958 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:57.912716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c6deb-f057-4573-ba0c-8bce11328958 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:57.914029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 671/1302] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:44:57.919190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd470b-bf6c-48b2-bff5-1c6d5b5b06d6 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:57.919601 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd470b-bf6c-48b2-bff5-1c6d5b5b06d6 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:57.919736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd470b-bf6c-48b2-bff5-1c6d5b5b06d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:57.937894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-28bd470b-bf6c-48b2-bff5-1c6d5b5b06d6 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:44:57.939616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-28bd470b-bf6c-48b2-bff5-1c6d5b5b06d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:44:58.005234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 632/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:44:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 11:44:58.274800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c6deb-f057-4573-ba0c-8bce11328958 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:58.372044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:58.378608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c6deb-f057-4573-ba0c-8bce11328958 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:58.413832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6f4d4-4e3d-4e9c-b7f9-7cd05d987f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:58.414451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6f4d4-4e3d-4e9c-b7f9-7cd05d987f8e None admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:58.414852 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6f4d4-4e3d-4e9c-b7f9-7cd05d987f8e None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:58.415130 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6f4d4-4e3d-4e9c-b7f9-7cd05d987f8e None admin] PATH_INFO: `/v3/services/placement` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:58.415716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6f4d4-4e3d-4e9c-b7f9-7cd05d987f8e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:58.417095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6f4d4-4e3d-4e9c-b7f9-7cd05d987f8e None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-27d6f4d4-4e3d-4e9c-b7f9-7cd05d987f8e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 11:44:58.425924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 11:44:58.427196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 11:44:58.428468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 672/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Oct 4 11:44:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 11:44:58.433059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd470b-bf6c-48b2-bff5-1c6d5b5b06d6 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:58.510684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:44:58.534179 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd470b-bf6c-48b2-bff5-1c6d5b5b06d6 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:58.567417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764f5d7-5938-4e51-8b79-a7e8e6effcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:58.568017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f5d7-5938-4e51-8b79-a7e8e6effcbd None admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:58.568342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f5d7-5938-4e51-8b79-a7e8e6effcbd None admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:58.568648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f5d7-5938-4e51-8b79-a7e8e6effcbd None admin] PATH_INFO: `/v3/services` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:58.569298 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764f5d7-5938-4e51-8b79-a7e8e6effcbd None admin] RBAC: Adding query filter params (name=placement) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:44:58.569603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764f5d7-5938-4e51-8b79-a7e8e6effcbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:58.571036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764f5d7-5938-4e51-8b79-a7e8e6effcbd None admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:58.576235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 633/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Oct 4 11:44:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:44:58.585327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6f4d4-4e3d-4e9c-b7f9-7cd05d987f8e None admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:44:58.684485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6f4d4-4e3d-4e9c-b7f9-7cd05d987f8e None admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:44:58.718677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31e861-8e49-47e3-b615-b40c8356d7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:44:58.719269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31e861-8e49-47e3-b615-b40c8356d7ff None admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:44:58.719574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31e861-8e49-47e3-b615-b40c8356d7ff None admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:44:58.719849 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31e861-8e49-47e3-b615-b40c8356d7ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:44:58.720543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31e861-8e49-47e3-b615-b40c8356d7ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:44:58.721902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31e861-8e49-47e3-b615-b40c8356d7ff None admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:44:58.794193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 673/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 11:44:58 2022] POST /identity/v3/endpoints => generated 331 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:45:08.573138 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=30 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:45:08.789261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:18.983140 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad2615-2193-4970-bfee-489cee002bc9 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:18.983752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad2615-2193-4970-bfee-489cee002bc9 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:18.983752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad2615-2193-4970-bfee-489cee002bc9 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:18.985976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 634/1307] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:53:18.991770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990958b5-6d2c-4322-8154-2725859ca5c8 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:18.992393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990958b5-6d2c-4322-8154-2725859ca5c8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:18.992682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990958b5-6d2c-4322-8154-2725859ca5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:19.119613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-990958b5-6d2c-4322-8154-2725859ca5c8 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:19.120893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-990958b5-6d2c-4322-8154-2725859ca5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:19.259378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 674/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:53:19.570724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f03a82-ac19-48b7-83cb-54319d33b947 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:19.571228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f03a82-ac19-48b7-83cb-54319d33b947 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:19.571711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f03a82-ac19-48b7-83cb-54319d33b947 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:19.574391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 635/1309] 10.222.0.15 () {54 vars in 964 bytes} [Tue Oct 4 11:53:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:53:19.581819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f9d1c-73c9-4fc0-a34c-bf38fc42774c None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:19.582219 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f9d1c-73c9-4fc0-a34c-bf38fc42774c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:19.582415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f9d1c-73c9-4fc0-a34c-bf38fc42774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:19.746047 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-0a6f9d1c-73c9-4fc0-a34c-bf38fc42774c None None] MFA Rules not processed for user `073e43c049224af29e9f6d426ea47e84`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:19.748980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-0a6f9d1c-73c9-4fc0-a34c-bf38fc42774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:19.791772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:19.882764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 675/1310] 10.222.0.15 () {58 vars in 1061 bytes} [Tue Oct 4 11:53:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:53:19.897429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f03a82-ac19-48b7-83cb-54319d33b947 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:53:19.899929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-72f03a82-ac19-48b7-83cb-54319d33b947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:20.144920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f03a82-ac19-48b7-83cb-54319d33b947 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:53:20.322391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be563bbe-b718-40d2-a398-fe5cd15b8634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:53:20.323677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be563bbe-b718-40d2-a398-fe5cd15b8634 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:20.324016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be563bbe-b718-40d2-a398-fe5cd15b8634 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:20.325067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be563bbe-b718-40d2-a398-fe5cd15b8634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:20.329772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-be563bbe-b718-40d2-a398-fe5cd15b8634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:20.477085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:20.595303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be563bbe-b718-40d2-a398-fe5cd15b8634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:53:20.597105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be563bbe-b718-40d2-a398-fe5cd15b8634 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:53:20.626528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 636/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:53:23.772521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d131e6-b0e9-4b11-b9c7-b2e556c80ef8 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:23.773105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d131e6-b0e9-4b11-b9c7-b2e556c80ef8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:23.773105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d131e6-b0e9-4b11-b9c7-b2e556c80ef8 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:23.774686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 676/1312] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:53:23.781498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b304-fa96-44c7-a8ba-6122ccc3e597 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:23.782173 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b304-fa96-44c7-a8ba-6122ccc3e597 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:23.782173 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b304-fa96-44c7-a8ba-6122ccc3e597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:23.872586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-15d3b304-fa96-44c7-a8ba-6122ccc3e597 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:23.874652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-15d3b304-fa96-44c7-a8ba-6122ccc3e597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:23.996618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 637/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:53:24.290659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d131e6-b0e9-4b11-b9c7-b2e556c80ef8 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:53:24.292357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-22d131e6-b0e9-4b11-b9c7-b2e556c80ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:24.415338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d131e6-b0e9-4b11-b9c7-b2e556c80ef8 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:53:24.495305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:24.499207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29068605-5ac2-45c6-84d5-8be1139292a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:53:24.499834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29068605-5ac2-45c6-84d5-8be1139292a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:24.500117 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29068605-5ac2-45c6-84d5-8be1139292a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:24.500431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29068605-5ac2-45c6-84d5-8be1139292a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:24.502244 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-29068605-5ac2-45c6-84d5-8be1139292a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:24.656889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29068605-5ac2-45c6-84d5-8be1139292a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:53:24.658411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29068605-5ac2-45c6-84d5-8be1139292a9 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:53:24.678881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 677/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:53:27.648045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14cbf0-f85f-4609-ad5c-9a18bb57de94 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:27.648630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14cbf0-f85f-4609-ad5c-9a18bb57de94 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:27.648630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14cbf0-f85f-4609-ad5c-9a18bb57de94 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:27.650027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 638/1315] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:53:27.657616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174653e-d916-4eb2-a871-d365074c8e9c None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:27.658131 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174653e-d916-4eb2-a871-d365074c8e9c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:27.658243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174653e-d916-4eb2-a871-d365074c8e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:27.770999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-5174653e-d916-4eb2-a871-d365074c8e9c None None] MFA Rules not processed for user `c0bf9cae9ba447d4851f472b6e46e3fd`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:27.773222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5174653e-d916-4eb2-a871-d365074c8e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:27.863214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:27.930127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 678/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:53:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:53:28.233576 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41de69a-f884-41a4-ae67-19dbb69d0a0f None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:28.233982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41de69a-f884-41a4-ae67-19dbb69d0a0f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:28.234364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41de69a-f884-41a4-ae67-19dbb69d0a0f None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:28.235847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 639/1317] 10.222.0.15 () {54 vars in 964 bytes} [Tue Oct 4 11:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:53:28.242460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbf173-2a5c-44e3-bb00-a1f4a0c85ec8 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:28.242827 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbf173-2a5c-44e3-bb00-a1f4a0c85ec8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:28.243025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbf173-2a5c-44e3-bb00-a1f4a0c85ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:28.332574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4dfbf173-2a5c-44e3-bb00-a1f4a0c85ec8 None None] MFA Rules not processed for user `073e43c049224af29e9f6d426ea47e84`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:28.333840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4dfbf173-2a5c-44e3-bb00-a1f4a0c85ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:28.419856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 679/1318] 10.222.0.15 () {58 vars in 1061 bytes} [Tue Oct 4 11:53:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:53:28.429325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41de69a-f884-41a4-ae67-19dbb69d0a0f None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:53:28.430931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f41de69a-f884-41a4-ae67-19dbb69d0a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:28.441529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:28.553266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41de69a-f884-41a4-ae67-19dbb69d0a0f None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:53:28.643220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dca3d1-219a-4e5d-8955-eda51489a3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:53:28.643847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dca3d1-219a-4e5d-8955-eda51489a3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:28.644105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dca3d1-219a-4e5d-8955-eda51489a3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:28.644404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dca3d1-219a-4e5d-8955-eda51489a3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:28.646264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-28dca3d1-219a-4e5d-8955-eda51489a3f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:28.723068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:28.779372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dca3d1-219a-4e5d-8955-eda51489a3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:53:28.780867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dca3d1-219a-4e5d-8955-eda51489a3f7 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:53:28.808277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 640/1319] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:53:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:53:32.479487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab3425-4a9b-4c38-84e4-0045c911924f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:32.480101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab3425-4a9b-4c38-84e4-0045c911924f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:32.480101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab3425-4a9b-4c38-84e4-0045c911924f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:32.481951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 680/1320] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:53:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:53:32.488001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f0709-598b-4973-9a2d-b8cb474b4688 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:32.488573 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f0709-598b-4973-9a2d-b8cb474b4688 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:32.488573 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f0709-598b-4973-9a2d-b8cb474b4688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:32.562309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-bb2f0709-598b-4973-9a2d-b8cb474b4688 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:32.563550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-bb2f0709-598b-4973-9a2d-b8cb474b4688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:32.697536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 641/1321] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:53:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:53:32.984246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab3425-4a9b-4c38-84e4-0045c911924f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:53:32.985831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-70ab3425-4a9b-4c38-84e4-0045c911924f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:33.046250 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:33.119286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab3425-4a9b-4c38-84e4-0045c911924f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:53:33.200510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d81766-62de-41f1-a0ad-69427ca0862e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:53:33.201128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d81766-62de-41f1-a0ad-69427ca0862e service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:33.201591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d81766-62de-41f1-a0ad-69427ca0862e service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:33.201808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d81766-62de-41f1-a0ad-69427ca0862e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:33.203540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e9d81766-62de-41f1-a0ad-69427ca0862e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:33.337214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:33.363808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d81766-62de-41f1-a0ad-69427ca0862e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:53:33.366673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d81766-62de-41f1-a0ad-69427ca0862e service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:53:33.409242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 681/1322] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:53:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:53:37.522539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606ddaa-58f0-4d05-8bf8-2be4f2194c8d None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:37.523441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606ddaa-58f0-4d05-8bf8-2be4f2194c8d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:37.523441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606ddaa-58f0-4d05-8bf8-2be4f2194c8d None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:37.524825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 642/1323] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:53:37.531428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f0ed1-87e8-4abf-84ab-161857baf9ba None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:37.532305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f0ed1-87e8-4abf-84ab-161857baf9ba None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:37.532305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f0ed1-87e8-4abf-84ab-161857baf9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:37.610790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-540f0ed1-87e8-4abf-84ab-161857baf9ba None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:37.612053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-540f0ed1-87e8-4abf-84ab-161857baf9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:37.731193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 682/1324] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:53:38.021850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606ddaa-58f0-4d05-8bf8-2be4f2194c8d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:53:38.023430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b606ddaa-58f0-4d05-8bf8-2be4f2194c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:38.046054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:38.146456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606ddaa-58f0-4d05-8bf8-2be4f2194c8d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:53:38.233794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012df14e-4d3d-455b-8c43-1b479aadd47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:53:38.234415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012df14e-4d3d-455b-8c43-1b479aadd47a service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:38.234695 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012df14e-4d3d-455b-8c43-1b479aadd47a service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:38.234976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012df14e-4d3d-455b-8c43-1b479aadd47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:38.236703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-012df14e-4d3d-455b-8c43-1b479aadd47a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:38.321995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:38.365347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012df14e-4d3d-455b-8c43-1b479aadd47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:53:38.366814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012df14e-4d3d-455b-8c43-1b479aadd47a service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:53:38.388676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 643/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:53:40.721054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda7463-add9-4c0c-9435-32201be58535 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:40.721054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda7463-add9-4c0c-9435-32201be58535 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:40.721816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda7463-add9-4c0c-9435-32201be58535 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:40.724280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 683/1326] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:53:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:53:40.729678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c1bda-82f9-4c73-994b-d7852895372d None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:40.730905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c1bda-82f9-4c73-994b-d7852895372d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:40.730905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c1bda-82f9-4c73-994b-d7852895372d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:40.836658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-700c1bda-82f9-4c73-994b-d7852895372d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:40.838010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-700c1bda-82f9-4c73-994b-d7852895372d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:40.958786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 644/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:53:41.252419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda7463-add9-4c0c-9435-32201be58535 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:53:41.255138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1fda7463-add9-4c0c-9435-32201be58535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:41.351102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:41.434123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda7463-add9-4c0c-9435-32201be58535 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:53:41.584306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be8b94-2b79-4a9e-99e3-bf918ef494db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:53:41.585118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be8b94-2b79-4a9e-99e3-bf918ef494db service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:41.585596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be8b94-2b79-4a9e-99e3-bf918ef494db service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:41.586065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be8b94-2b79-4a9e-99e3-bf918ef494db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:41.589289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-71be8b94-2b79-4a9e-99e3-bf918ef494db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:41.812204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:41.816444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be8b94-2b79-4a9e-99e3-bf918ef494db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:53:41.819247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be8b94-2b79-4a9e-99e3-bf918ef494db service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:53:41.856611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 684/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:53:44.216428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7d7d4-8911-4ea3-85e7-d2691eca107a None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:44.217004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7d7d4-8911-4ea3-85e7-d2691eca107a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:44.217004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7d7d4-8911-4ea3-85e7-d2691eca107a None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:44.218370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 645/1329] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:53:44.225620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466d716-33a9-4bd1-9c93-6023bfdc707f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:44.226191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466d716-33a9-4bd1-9c93-6023bfdc707f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:44.226610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466d716-33a9-4bd1-9c93-6023bfdc707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:44.358598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2466d716-33a9-4bd1-9c93-6023bfdc707f None None] MFA Rules not processed for user `c0bf9cae9ba447d4851f472b6e46e3fd`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:44.360783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2466d716-33a9-4bd1-9c93-6023bfdc707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:44.545458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 685/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:53:44 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:53:44.846052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7d7d4-8911-4ea3-85e7-d2691eca107a None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:53:44.849023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-05d7d7d4-8911-4ea3-85e7-d2691eca107a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:44.910347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:45.024635 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7d7d4-8911-4ea3-85e7-d2691eca107a None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:53:45.123882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309544e6-910a-41d7-81a3-fc423fe6d14c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:53:45.124550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309544e6-910a-41d7-81a3-fc423fe6d14c service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:45.124821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309544e6-910a-41d7-81a3-fc423fe6d14c service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:45.125093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309544e6-910a-41d7-81a3-fc423fe6d14c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:45.126830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-309544e6-910a-41d7-81a3-fc423fe6d14c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:45.225717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:45.281695 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309544e6-910a-41d7-81a3-fc423fe6d14c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:53:45.285188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309544e6-910a-41d7-81a3-fc423fe6d14c service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:53:45.326675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 646/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:53:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:53:48.257296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0b2d3-78d9-47ad-aa11-7427796bb8f9 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:48.258037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0b2d3-78d9-47ad-aa11-7427796bb8f9 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:48.258037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0b2d3-78d9-47ad-aa11-7427796bb8f9 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:48.259212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 686/1332] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:53:48.265524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62537bb1-c70d-4a96-919e-fc5da1eeee26 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:48.266136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62537bb1-c70d-4a96-919e-fc5da1eeee26 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:48.266136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62537bb1-c70d-4a96-919e-fc5da1eeee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:48.342474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-62537bb1-c70d-4a96-919e-fc5da1eeee26 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:48.343767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-62537bb1-c70d-4a96-919e-fc5da1eeee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:48.455371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 647/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:53:48.741896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0b2d3-78d9-47ad-aa11-7427796bb8f9 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:53:48.743473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6cb0b2d3-78d9-47ad-aa11-7427796bb8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:48.829140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:48.866999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0b2d3-78d9-47ad-aa11-7427796bb8f9 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:53:48.962716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7b0f-0cfe-4843-b49d-5f1f4b8f3ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:53:48.963330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7b0f-0cfe-4843-b49d-5f1f4b8f3ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:48.963614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7b0f-0cfe-4843-b49d-5f1f4b8f3ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:48.963889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7b0f-0cfe-4843-b49d-5f1f4b8f3ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:48.965642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-55ac7b0f-0cfe-4843-b49d-5f1f4b8f3ea6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:49.097209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac7b0f-0cfe-4843-b49d-5f1f4b8f3ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:53:49.098667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac7b0f-0cfe-4843-b49d-5f1f4b8f3ea6 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:53:49.111269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:49.120813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 687/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:53:51.449589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f7b8d-941c-4063-b4e8-b4e15f2ac469 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:51.450207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f7b8d-941c-4063-b4e8-b4e15f2ac469 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:51.450207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f7b8d-941c-4063-b4e8-b4e15f2ac469 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:51.451495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 648/1335] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:53:51.457415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55673d-3a21-4e3b-8974-ef1574257da1 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:51.458266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55673d-3a21-4e3b-8974-ef1574257da1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:51.458431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55673d-3a21-4e3b-8974-ef1574257da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:51.533990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-cc55673d-3a21-4e3b-8974-ef1574257da1 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:51.535260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cc55673d-3a21-4e3b-8974-ef1574257da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:51.646109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 688/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:53:51.942144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f7b8d-941c-4063-b4e8-b4e15f2ac469 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:53:51.943713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a38f7b8d-941c-4063-b4e8-b4e15f2ac469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:51.991032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:52.070791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f7b8d-941c-4063-b4e8-b4e15f2ac469 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:53:52.157673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5394b-7652-437f-beea-c244222b5c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:53:52.158303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5394b-7652-437f-beea-c244222b5c5e service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:52.158581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5394b-7652-437f-beea-c244222b5c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:52.158926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5394b-7652-437f-beea-c244222b5c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:52.160612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3da5394b-7652-437f-beea-c244222b5c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:52.259579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:52.289627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5394b-7652-437f-beea-c244222b5c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:53:52.291242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5394b-7652-437f-beea-c244222b5c5e service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:53:52.313707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 649/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:53:55.359935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970cb60-c2c3-44c7-8349-82cbb19d91d5 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:55.360579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970cb60-c2c3-44c7-8349-82cbb19d91d5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:55.360579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970cb60-c2c3-44c7-8349-82cbb19d91d5 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:55.361959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 689/1338] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:53:55.368108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da1e69-7184-4e58-9759-4e61196cfd5b None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:55.368856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da1e69-7184-4e58-9759-4e61196cfd5b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:55.369134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da1e69-7184-4e58-9759-4e61196cfd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:55.449122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-12da1e69-7184-4e58-9759-4e61196cfd5b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:53:55.450394 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-12da1e69-7184-4e58-9759-4e61196cfd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:55.566498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 650/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:53:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:53:55.856705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970cb60-c2c3-44c7-8349-82cbb19d91d5 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:53:55.858289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d970cb60-c2c3-44c7-8349-82cbb19d91d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:55.951538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:53:55.985756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970cb60-c2c3-44c7-8349-82cbb19d91d5 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:53:56.079522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460fe14-90ba-4b1c-a465-4540158326b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:53:56.080031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460fe14-90ba-4b1c-a465-4540158326b2 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:53:56.080345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460fe14-90ba-4b1c-a465-4540158326b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:53:56.080620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460fe14-90ba-4b1c-a465-4540158326b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:53:56.082385 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5460fe14-90ba-4b1c-a465-4540158326b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:53:56.228356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460fe14-90ba-4b1c-a465-4540158326b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:53:56.231090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460fe14-90ba-4b1c-a465-4540158326b2 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:53:56.268360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 690/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:53:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:54:01.365632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2772235-9b03-4152-b47d-70c3794872db None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:01.366295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2772235-9b03-4152-b47d-70c3794872db None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:01.366295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2772235-9b03-4152-b47d-70c3794872db None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:01.367780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 651/1341] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:01.373722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc1ac2-734a-47bf-bc9f-ab1aa394eeb4 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:01.374795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc1ac2-734a-47bf-bc9f-ab1aa394eeb4 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:01.374795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc1ac2-734a-47bf-bc9f-ab1aa394eeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:01.380394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:01.459293 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-52fc1ac2-734a-47bf-bc9f-ab1aa394eeb4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:01.460595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-52fc1ac2-734a-47bf-bc9f-ab1aa394eeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:01.585553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 691/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:54:01.880339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2772235-9b03-4152-b47d-70c3794872db None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:01.882495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c2772235-9b03-4152-b47d-70c3794872db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:01.959660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:02.036594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2772235-9b03-4152-b47d-70c3794872db None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:02.126704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1e022-db91-4c0b-8158-01e28481acb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:02.127278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1e022-db91-4c0b-8158-01e28481acb0 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:02.127784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1e022-db91-4c0b-8158-01e28481acb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:02.127893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1e022-db91-4c0b-8158-01e28481acb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:02.129867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-93d1e022-db91-4c0b-8158-01e28481acb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:02.242177 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:02.263151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1e022-db91-4c0b-8158-01e28481acb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:02.264938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1e022-db91-4c0b-8158-01e28481acb0 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:02.286840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 652/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:54:05.951129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cadfe9-5df4-4d9d-8f86-5cb75e7cb882 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:05.951986 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cadfe9-5df4-4d9d-8f86-5cb75e7cb882 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:05.952164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cadfe9-5df4-4d9d-8f86-5cb75e7cb882 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:05.956210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 692/1344] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:05.963035 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bfc50-f8c4-4a19-a2ac-47ccb2e4d29d None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:05.963664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bfc50-f8c4-4a19-a2ac-47ccb2e4d29d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:05.964120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bfc50-f8c4-4a19-a2ac-47ccb2e4d29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:06.080710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-060bfc50-f8c4-4a19-a2ac-47ccb2e4d29d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:06.080710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-060bfc50-f8c4-4a19-a2ac-47ccb2e4d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:06.190344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 653/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:06.480114 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cadfe9-5df4-4d9d-8f86-5cb75e7cb882 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:06.481879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-85cadfe9-5df4-4d9d-8f86-5cb75e7cb882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:06.595839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:06.627173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cadfe9-5df4-4d9d-8f86-5cb75e7cb882 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:06.739505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda44428-4bdc-4886-b8fb-7dfd77cccdfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:06.740413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda44428-4bdc-4886-b8fb-7dfd77cccdfe service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:06.740841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda44428-4bdc-4886-b8fb-7dfd77cccdfe service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:06.741254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda44428-4bdc-4886-b8fb-7dfd77cccdfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:06.743717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-bda44428-4bdc-4886-b8fb-7dfd77cccdfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:06.929945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda44428-4bdc-4886-b8fb-7dfd77cccdfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:06.932597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda44428-4bdc-4886-b8fb-7dfd77cccdfe service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:06.969877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 693/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:54:12.569230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f200cf-18ec-4dcc-8add-89f7c6b0f451 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:12.569888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f200cf-18ec-4dcc-8add-89f7c6b0f451 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:12.569888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f200cf-18ec-4dcc-8add-89f7c6b0f451 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:12.571045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 654/1347] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:54:12.578112 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98b837-4b17-4e67-92b0-6b25bfe10424 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:12.578712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98b837-4b17-4e67-92b0-6b25bfe10424 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:12.578829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98b837-4b17-4e67-92b0-6b25bfe10424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:12.602877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:12.698997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2a98b837-4b17-4e67-92b0-6b25bfe10424 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:12.700600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2a98b837-4b17-4e67-92b0-6b25bfe10424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:12.857900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 694/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:13.188297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f200cf-18ec-4dcc-8add-89f7c6b0f451 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:13.189988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e8f200cf-18ec-4dcc-8add-89f7c6b0f451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:13.262409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:13.328013 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f200cf-18ec-4dcc-8add-89f7c6b0f451 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:13.436864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936c19b-e0a2-49cb-b4a2-87159d87ed4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:13.437599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936c19b-e0a2-49cb-b4a2-87159d87ed4a service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:13.438052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936c19b-e0a2-49cb-b4a2-87159d87ed4a service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:13.438436 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936c19b-e0a2-49cb-b4a2-87159d87ed4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:13.441234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4936c19b-e0a2-49cb-b4a2-87159d87ed4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:13.578179 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:13.582122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936c19b-e0a2-49cb-b4a2-87159d87ed4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:13.583623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936c19b-e0a2-49cb-b4a2-87159d87ed4a service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:13.605022 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 655/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:54:19.492677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070ae63-7446-4478-a4a8-478a3921c739 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:19.493590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070ae63-7446-4478-a4a8-478a3921c739 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:19.494045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070ae63-7446-4478-a4a8-478a3921c739 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:19.496049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 695/1350] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 11:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:19.516990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71596efa-23b5-4ba5-b5da-a5d53fdeb44b None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:19.517771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71596efa-23b5-4ba5-b5da-a5d53fdeb44b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:19.524471 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71596efa-23b5-4ba5-b5da-a5d53fdeb44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:19.712933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-71596efa-23b5-4ba5-b5da-a5d53fdeb44b None None] MFA Rules not processed for user `16374dbb6d084ddfa3b49afd19c7c325`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:19.715038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-71596efa-23b5-4ba5-b5da-a5d53fdeb44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:20.076878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 656/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 11:54:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 561 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:22.855915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=59 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:24.437594 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74393d-d85a-43c0-b515-8d9007a1dd22 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:24.438504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74393d-d85a-43c0-b515-8d9007a1dd22 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:24.438504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74393d-d85a-43c0-b515-8d9007a1dd22 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:24.439900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 696/1352] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 11:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:54:24.445633 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091b00b-cdb9-495f-a50b-f03dce524e7c None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:24.446643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091b00b-cdb9-495f-a50b-f03dce524e7c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:24.446917 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091b00b-cdb9-495f-a50b-f03dce524e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:24.549562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:24.596823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d091b00b-cdb9-495f-a50b-f03dce524e7c None None] MFA Rules not processed for user `16374dbb6d084ddfa3b49afd19c7c325`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:24.600060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d091b00b-cdb9-495f-a50b-f03dce524e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:24.814674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 657/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 11:54:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:27.132877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43129bac-f785-4678-919f-c60e11aefea1 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:27.133703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43129bac-f785-4678-919f-c60e11aefea1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:27.133703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43129bac-f785-4678-919f-c60e11aefea1 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:27.135339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 697/1354] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 11:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:27.141814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd95ff-f93b-485f-b524-c53b89179704 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:27.142498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd95ff-f93b-485f-b524-c53b89179704 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:27.142498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd95ff-f93b-485f-b524-c53b89179704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:27.237379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f6dd95ff-f93b-485f-b524-c53b89179704 None None] MFA Rules not processed for user `16374dbb6d084ddfa3b49afd19c7c325`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:27.238726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f6dd95ff-f93b-485f-b524-c53b89179704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:27.293715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:27.381985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 658/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 11:54:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:54:31.259920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44029589-4e92-4aab-a97c-9cc81583c6c7 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:31.260534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44029589-4e92-4aab-a97c-9cc81583c6c7 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:31.261013 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44029589-4e92-4aab-a97c-9cc81583c6c7 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:31.264156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 698/1356] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:31.270384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add8904-d7a7-4545-b570-1e0f0e97711e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:31.271213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add8904-d7a7-4545-b570-1e0f0e97711e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:31.271622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add8904-d7a7-4545-b570-1e0f0e97711e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:31.372019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-6add8904-d7a7-4545-b570-1e0f0e97711e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:31.374471 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-6add8904-d7a7-4545-b570-1e0f0e97711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:31.615118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 659/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:54:32.128812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb0f4d-f1c3-4ab4-a3c8-e7e5762148ab None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:32.129825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb0f4d-f1c3-4ab4-a3c8-e7e5762148ab None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:32.130942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb0f4d-f1c3-4ab4-a3c8-e7e5762148ab None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:32.137027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 699/1358] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 4 11:54:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:32.152771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f429677-a1bc-4740-92a4-054e466a7016 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:32.154000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f429677-a1bc-4740-92a4-054e466a7016 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:32.155378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f429677-a1bc-4740-92a4-054e466a7016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:32.189949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:32.315215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-0f429677-a1bc-4740-92a4-054e466a7016 None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:32.317228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0f429677-a1bc-4740-92a4-054e466a7016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:32.565164 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 660/1359] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 4 11:54:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:32.579134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb0f4d-f1c3-4ab4-a3c8-e7e5762148ab None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:32.583213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e2eb0f4d-f1c3-4ab4-a3c8-e7e5762148ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:32.865628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb0f4d-f1c3-4ab4-a3c8-e7e5762148ab None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:33.053736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:33.111874 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e6260-2c9e-47ec-81e2-f472d19d685d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:33.112686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e6260-2c9e-47ec-81e2-f472d19d685d service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:33.113008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e6260-2c9e-47ec-81e2-f472d19d685d service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:33.113352 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e6260-2c9e-47ec-81e2-f472d19d685d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:33.115279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-be6e6260-2c9e-47ec-81e2-f472d19d685d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:33.246343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e6260-2c9e-47ec-81e2-f472d19d685d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:33.249732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e6260-2c9e-47ec-81e2-f472d19d685d service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:33.289072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 700/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:54:36.143557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58fe26-0c17-456d-b098-d6ab87c8ae4c None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:36.144598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58fe26-0c17-456d-b098-d6ab87c8ae4c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:36.144598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58fe26-0c17-456d-b098-d6ab87c8ae4c None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:36.145528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 661/1361] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:54:36.151778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25054d-a606-4df0-bade-327ec6cda29b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:36.152754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25054d-a606-4df0-bade-327ec6cda29b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:36.152754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25054d-a606-4df0-bade-327ec6cda29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:36.236757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1d25054d-a606-4df0-bade-327ec6cda29b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:36.238001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1d25054d-a606-4df0-bade-327ec6cda29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:36.289752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:36.362124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 701/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:54:36.662325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58fe26-0c17-456d-b098-d6ab87c8ae4c None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:36.664087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cb58fe26-0c17-456d-b098-d6ab87c8ae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:36.747058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:36.933362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58fe26-0c17-456d-b098-d6ab87c8ae4c None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:37.142881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d13138-362c-46b0-836e-374424e75f3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:37.143088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d13138-362c-46b0-836e-374424e75f3f service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:37.144218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d13138-362c-46b0-836e-374424e75f3f service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:37.144218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d13138-362c-46b0-836e-374424e75f3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:37.145450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b7d13138-362c-46b0-836e-374424e75f3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:37.178963 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:37.297862 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d13138-362c-46b0-836e-374424e75f3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:37.302480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d13138-362c-46b0-836e-374424e75f3f service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:37.326365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 662/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:54:40.709734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86969a3a-d3d9-4b4e-8699-88c15324fc1f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:40.710553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86969a3a-d3d9-4b4e-8699-88c15324fc1f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:40.711194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86969a3a-d3d9-4b4e-8699-88c15324fc1f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:40.714925 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 702/1364] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:40.722294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b996bc-1d0d-40c5-a9e7-5eed7b831bbe None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:40.723262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b996bc-1d0d-40c5-a9e7-5eed7b831bbe None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:40.723885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b996bc-1d0d-40c5-a9e7-5eed7b831bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:40.876151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-96b996bc-1d0d-40c5-a9e7-5eed7b831bbe None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:40.877855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-96b996bc-1d0d-40c5-a9e7-5eed7b831bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:41.000363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:41.012773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 663/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:41.311235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec2ecc-e675-4ee2-b8ff-26b0c621eafb None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:41.311527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec2ecc-e675-4ee2-b8ff-26b0c621eafb None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:41.311819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec2ecc-e675-4ee2-b8ff-26b0c621eafb None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:41.313889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 703/1366] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 4 11:54:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:41.320474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348611b-0ec4-4e82-853e-7db29d4d1db3 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:41.320830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348611b-0ec4-4e82-853e-7db29d4d1db3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:41.320993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348611b-0ec4-4e82-853e-7db29d4d1db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:41.435836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f348611b-0ec4-4e82-853e-7db29d4d1db3 None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:41.437533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f348611b-0ec4-4e82-853e-7db29d4d1db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:41.501913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74fda-398d-4313-8a55-da1ce15b4395 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:41.502359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74fda-398d-4313-8a55-da1ce15b4395 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:41.502899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74fda-398d-4313-8a55-da1ce15b4395 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:41.506122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 704/1367] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:41.513275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd591022-bbf4-43ca-bf24-2b769604b475 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:41.513787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd591022-bbf4-43ca-bf24-2b769604b475 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:41.514285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd591022-bbf4-43ca-bf24-2b769604b475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:41.583561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 664/1368] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 4 11:54:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:41.604320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348611b-0ec4-4e82-853e-7db29d4d1db3 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:41.619275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-fd591022-bbf4-43ca-bf24-2b769604b475 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:41.621206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fd591022-bbf4-43ca-bf24-2b769604b475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:41.622782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348611b-0ec4-4e82-853e-7db29d4d1db3 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:41.696604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:41.769529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 705/1369] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:41.811545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291433c-696f-43bb-b501-f3d51e949f84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:41.812352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291433c-696f-43bb-b501-f3d51e949f84 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:41.812735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291433c-696f-43bb-b501-f3d51e949f84 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:41.813115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291433c-696f-43bb-b501-f3d51e949f84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:41.814959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4291433c-696f-43bb-b501-f3d51e949f84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:41.928587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291433c-696f-43bb-b501-f3d51e949f84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:41.930495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291433c-696f-43bb-b501-f3d51e949f84 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:41.954367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 665/1370] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:54:42.075619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ee33d-3005-4688-a980-86ee6c4b018f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:42.076211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ee33d-3005-4688-a980-86ee6c4b018f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:42.076734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ee33d-3005-4688-a980-86ee6c4b018f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:42.078263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 706/1371] 10.222.0.15 () {54 vars in 963 bytes} [Tue Oct 4 11:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:42.085660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3004b3a-730a-4f82-b9cd-a034f8a7d8ce None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:42.085983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3004b3a-730a-4f82-b9cd-a034f8a7d8ce None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:42.086215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3004b3a-730a-4f82-b9cd-a034f8a7d8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:42.161535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:42.187184 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b3004b3a-730a-4f82-b9cd-a034f8a7d8ce None None] MFA Rules not processed for user `496a2e8bf591437888ea82161ca4d5ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:42.189452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b3004b3a-730a-4f82-b9cd-a034f8a7d8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:42.286844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 666/1372] 10.222.0.15 () {58 vars in 1060 bytes} [Tue Oct 4 11:54:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:42.301026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ee33d-3005-4688-a980-86ee6c4b018f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:42.302584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-483ee33d-3005-4688-a980-86ee6c4b018f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:42.327808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:42.465810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ee33d-3005-4688-a980-86ee6c4b018f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:42.641278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d8c12-5ded-4f77-8df8-10327e3b02fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:42.642340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d8c12-5ded-4f77-8df8-10327e3b02fe service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:42.642520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d8c12-5ded-4f77-8df8-10327e3b02fe service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:42.643074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d8c12-5ded-4f77-8df8-10327e3b02fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:42.645772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c34d8c12-5ded-4f77-8df8-10327e3b02fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:42.758228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:42.855575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d8c12-5ded-4f77-8df8-10327e3b02fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:42.857237 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d8c12-5ded-4f77-8df8-10327e3b02fe service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:42.888070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 707/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:54:44.657218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08dc7a-55a4-4b5e-9de5-2dab952e8334 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:44.658044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08dc7a-55a4-4b5e-9de5-2dab952e8334 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:44.658044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08dc7a-55a4-4b5e-9de5-2dab952e8334 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:44.660258 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 667/1374] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:44.668313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ef450-eb50-407f-bee7-ee50052cf0c2 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:44.669074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ef450-eb50-407f-bee7-ee50052cf0c2 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:44.669578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ef450-eb50-407f-bee7-ee50052cf0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:44.775753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-6c1ef450-eb50-407f-bee7-ee50052cf0c2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:44.778040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6c1ef450-eb50-407f-bee7-ee50052cf0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:44.934862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 708/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:45.267042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08dc7a-55a4-4b5e-9de5-2dab952e8334 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:45.268839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5d08dc7a-55a4-4b5e-9de5-2dab952e8334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:45.468679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:45.474679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08dc7a-55a4-4b5e-9de5-2dab952e8334 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:45.641608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83749ebe-1b2d-4527-900b-b8a41ab3412a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:45.642258 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83749ebe-1b2d-4527-900b-b8a41ab3412a service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:45.642613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83749ebe-1b2d-4527-900b-b8a41ab3412a service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:45.642892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83749ebe-1b2d-4527-900b-b8a41ab3412a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:45.644691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-83749ebe-1b2d-4527-900b-b8a41ab3412a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:45.793790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83749ebe-1b2d-4527-900b-b8a41ab3412a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:45.797115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83749ebe-1b2d-4527-900b-b8a41ab3412a service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:45.824152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:45.829871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 668/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:54:48.841553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eaba0-2576-41b2-bf8e-176e305ee7ef None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:48.842195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eaba0-2576-41b2-bf8e-176e305ee7ef None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:48.842529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eaba0-2576-41b2-bf8e-176e305ee7ef None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:48.845099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 709/1377] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:48.853712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9a9a8-933a-4bee-ab1a-175022bf0a8e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:48.854578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9a9a8-933a-4bee-ab1a-175022bf0a8e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:48.854578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9a9a8-933a-4bee-ab1a-175022bf0a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:48.961663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-14f9a9a8-933a-4bee-ab1a-175022bf0a8e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:48.963115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-14f9a9a8-933a-4bee-ab1a-175022bf0a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:49.100346 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 669/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:49.448916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15eaba0-2576-41b2-bf8e-176e305ee7ef None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:49.450506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d15eaba0-2576-41b2-bf8e-176e305ee7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:49.493750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:49.725713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15eaba0-2576-41b2-bf8e-176e305ee7ef None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:49.837998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:49.851362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8465c-234e-4dea-9653-5457becd91cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:49.852461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8465c-234e-4dea-9653-5457becd91cf service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:49.852461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8465c-234e-4dea-9653-5457becd91cf service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:49.852823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8465c-234e-4dea-9653-5457becd91cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:49.854337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a0f8465c-234e-4dea-9653-5457becd91cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:49.984563 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8465c-234e-4dea-9653-5457becd91cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:49.986165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8465c-234e-4dea-9653-5457becd91cf service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:50.010122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 710/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:54:52.592129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c366c-7bfe-4824-82f8-a434356d40af None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:52.592755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c366c-7bfe-4824-82f8-a434356d40af None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:52.592858 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c366c-7bfe-4824-82f8-a434356d40af None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:52.594486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 670/1380] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:52.601262 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ea642-2ff5-413c-bc6f-1435c3967a3d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:52.602556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ea642-2ff5-413c-bc6f-1435c3967a3d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:52.602556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ea642-2ff5-413c-bc6f-1435c3967a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:52.693268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-349ea642-2ff5-413c-bc6f-1435c3967a3d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:52.694553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-349ea642-2ff5-413c-bc6f-1435c3967a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:52.768476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:52.826833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 711/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:54:53.151301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c366c-7bfe-4824-82f8-a434356d40af None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:53.153325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d38c366c-7bfe-4824-82f8-a434356d40af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:53.291390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:53.403144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c366c-7bfe-4824-82f8-a434356d40af None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:53.576118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ec6ee-f4d6-480e-9170-17da005a6b8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:53.576738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ec6ee-f4d6-480e-9170-17da005a6b8b service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:53.577133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ec6ee-f4d6-480e-9170-17da005a6b8b service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:53.577622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ec6ee-f4d6-480e-9170-17da005a6b8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:53.579669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-844ec6ee-f4d6-480e-9170-17da005a6b8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:53.647811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:53.745921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ec6ee-f4d6-480e-9170-17da005a6b8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:53.750348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ec6ee-f4d6-480e-9170-17da005a6b8b service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:53.780216 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 671/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:54:57.806362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f14581-0ea2-4455-bf30-6f483ae4ec7a None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:57.806984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f14581-0ea2-4455-bf30-6f483ae4ec7a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:57.807257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f14581-0ea2-4455-bf30-6f483ae4ec7a None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:57.809968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 712/1383] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:54:57.818014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fd6e7-9237-4aa2-b167-2f0514fb66a0 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:57.818804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fd6e7-9237-4aa2-b167-2f0514fb66a0 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:57.818950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fd6e7-9237-4aa2-b167-2f0514fb66a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:57.906538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-288fd6e7-9237-4aa2-b167-2f0514fb66a0 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:54:57.907842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-288fd6e7-9237-4aa2-b167-2f0514fb66a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:58.055447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 672/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:54:58.363434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f14581-0ea2-4455-bf30-6f483ae4ec7a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:54:58.365216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-76f14581-0ea2-4455-bf30-6f483ae4ec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:58.609468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f14581-0ea2-4455-bf30-6f483ae4ec7a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:54:58.678398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:54:58.805361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4c4fb-d1e2-457c-9e16-a56fd5d906cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:54:58.806162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c4fb-d1e2-457c-9e16-a56fd5d906cb service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:58.806592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c4fb-d1e2-457c-9e16-a56fd5d906cb service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:58.807525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c4fb-d1e2-457c-9e16-a56fd5d906cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:58.808835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e5d4c4fb-d1e2-457c-9e16-a56fd5d906cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:54:58.963435 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4c4fb-d1e2-457c-9e16-a56fd5d906cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:54:58.966551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4c4fb-d1e2-457c-9e16-a56fd5d906cb service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:54:59.004135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 713/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:54:59.962519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa436bce-4c8b-4f09-b78d-7a91ffdf7faa None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:59.963319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa436bce-4c8b-4f09-b78d-7a91ffdf7faa None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:59.963319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa436bce-4c8b-4f09-b78d-7a91ffdf7faa None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:54:59.964831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 673/1386] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:54:59.972464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9232-3311-4779-ab0c-192685574562 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:54:59.973085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9232-3311-4779-ab0c-192685574562 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:54:59.973480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9232-3311-4779-ab0c-192685574562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:00.086623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-811f9232-3311-4779-ab0c-192685574562 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:00.089272 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-811f9232-3311-4779-ab0c-192685574562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:00.093770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:00.274879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 714/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:00.596787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35c8d4-36bf-4dfe-8e96-5441a010dadd None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:00.597267 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35c8d4-36bf-4dfe-8e96-5441a010dadd None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:00.597748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35c8d4-36bf-4dfe-8e96-5441a010dadd None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:00.600472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 674/1388] 10.222.0.15 () {54 vars in 962 bytes} [Tue Oct 4 11:55:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:00.612462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd840f-a218-417e-a08b-962bc45f1b0e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:00.612462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd840f-a218-417e-a08b-962bc45f1b0e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:00.613126 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd840f-a218-417e-a08b-962bc45f1b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:00.723087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f0dd840f-a218-417e-a08b-962bc45f1b0e None None] MFA Rules not processed for user `454fb811f6e34e1d881d4948ede058b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:00.724381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f0dd840f-a218-417e-a08b-962bc45f1b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:00.841331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:00.845447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 715/1389] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Oct 4 11:55:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:00.859783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35c8d4-36bf-4dfe-8e96-5441a010dadd None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:00.861852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2f35c8d4-36bf-4dfe-8e96-5441a010dadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:00.893749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:01.025655 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35c8d4-36bf-4dfe-8e96-5441a010dadd None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:01.167976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88d03c-528b-40d1-9534-6a616c61e0d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454fb811f6e34e1d881d4948ede058b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:01.168933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d03c-528b-40d1-9534-6a616c61e0d1 service glance] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:01.169397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d03c-528b-40d1-9534-6a616c61e0d1 service glance] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:01.170030 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88d03c-528b-40d1-9534-6a616c61e0d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:01.173913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5f88d03c-528b-40d1-9534-6a616c61e0d1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:01.325887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:01.401913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88d03c-528b-40d1-9534-6a616c61e0d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:01.403548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88d03c-528b-40d1-9534-6a616c61e0d1 service glance] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:01.432850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 675/1390] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:01.497186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bcd1d-e249-45fc-b0d6-1c7afd09eba5 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:01.497741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bcd1d-e249-45fc-b0d6-1c7afd09eba5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:01.497997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bcd1d-e249-45fc-b0d6-1c7afd09eba5 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:01.499841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 716/1391] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:55:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:01.506735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512caa8b-d2a6-4c17-91b2-c64ca42962c8 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:01.507049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512caa8b-d2a6-4c17-91b2-c64ca42962c8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:01.507363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512caa8b-d2a6-4c17-91b2-c64ca42962c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:01.590394 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-512caa8b-d2a6-4c17-91b2-c64ca42962c8 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:01.591654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-512caa8b-d2a6-4c17-91b2-c64ca42962c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:01.709180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 676/1392] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:55:02.009571 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bcd1d-e249-45fc-b0d6-1c7afd09eba5 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:02.011136 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e05bcd1d-e249-45fc-b0d6-1c7afd09eba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:02.197397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bcd1d-e249-45fc-b0d6-1c7afd09eba5 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:02.302227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:02.390922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea26d25-9e59-4202-a76a-1817beab7d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:02.391953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea26d25-9e59-4202-a76a-1817beab7d5a service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:02.392519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea26d25-9e59-4202-a76a-1817beab7d5a service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:02.393006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea26d25-9e59-4202-a76a-1817beab7d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:02.396221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-bea26d25-9e59-4202-a76a-1817beab7d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:02.575894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea26d25-9e59-4202-a76a-1817beab7d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:02.578968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea26d25-9e59-4202-a76a-1817beab7d5a service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:02.616845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 717/1393] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:55:02.741575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555df2a-edd8-425b-86ef-1d5f8cb17304 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:02.742227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555df2a-edd8-425b-86ef-1d5f8cb17304 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:02.742227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555df2a-edd8-425b-86ef-1d5f8cb17304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:02.791970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:02.830829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b555df2a-edd8-425b-86ef-1d5f8cb17304 None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:02.832088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b555df2a-edd8-425b-86ef-1d5f8cb17304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:02.951003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 677/1394] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 11:55:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:03.025544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26619b6e-8c99-4e9e-9932-eb53a2b3d310 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:03.025941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26619b6e-8c99-4e9e-9932-eb53a2b3d310 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:03.026374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26619b6e-8c99-4e9e-9932-eb53a2b3d310 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:03.028573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 718/1395] 10.222.0.15 () {54 vars in 943 bytes} [Tue Oct 4 11:55:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:03.037823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99350577-3fde-4643-adbb-9f9f512a2890 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:03.038603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99350577-3fde-4643-adbb-9f9f512a2890 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:03.039174 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99350577-3fde-4643-adbb-9f9f512a2890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:03.160319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-99350577-3fde-4643-adbb-9f9f512a2890 None None] MFA Rules not processed for user `734d8f03ff2949959cbfce98212a3ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:03.161588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-99350577-3fde-4643-adbb-9f9f512a2890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:03.234709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:03.312447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 678/1396] 10.222.0.15 () {58 vars in 1040 bytes} [Tue Oct 4 11:55:03 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:03.322414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26619b6e-8c99-4e9e-9932-eb53a2b3d310 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:03.324004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-26619b6e-8c99-4e9e-9932-eb53a2b3d310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:03.438241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:03.515309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26619b6e-8c99-4e9e-9932-eb53a2b3d310 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:03.655738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884e0d6-7524-45ec-8129-1b217d03df97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:03.656883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884e0d6-7524-45ec-8129-1b217d03df97 service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:03.656883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884e0d6-7524-45ec-8129-1b217d03df97 service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:03.657215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884e0d6-7524-45ec-8129-1b217d03df97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:03.659060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8884e0d6-7524-45ec-8129-1b217d03df97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:03.759811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:03.781158 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884e0d6-7524-45ec-8129-1b217d03df97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:03.783087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884e0d6-7524-45ec-8129-1b217d03df97 service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:03.799537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 719/1397] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 11:55:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 11:55:05.775631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8b92-076e-4821-b2d2-c99ccecd3456 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:05.776939 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8b92-076e-4821-b2d2-c99ccecd3456 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:05.777248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8b92-076e-4821-b2d2-c99ccecd3456 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:05.778766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 679/1398] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:55:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:55:05.788704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b7d8f-6fd0-438d-8e3e-443aebeb3ad6 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:05.790767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b7d8f-6fd0-438d-8e3e-443aebeb3ad6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:05.790767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b7d8f-6fd0-438d-8e3e-443aebeb3ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:05.889626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-905b7d8f-6fd0-438d-8e3e-443aebeb3ad6 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:05.890927 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-905b7d8f-6fd0-438d-8e3e-443aebeb3ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:06.103255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 720/1399] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:06.443126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c8b92-076e-4821-b2d2-c99ccecd3456 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:06.446154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-9c8c8b92-076e-4821-b2d2-c99ccecd3456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:06.758177 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c8b92-076e-4821-b2d2-c99ccecd3456 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:06.818517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:07.012383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59aa62a-2957-4466-822f-57fa185afa8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:07.013351 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59aa62a-2957-4466-822f-57fa185afa8c service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:07.013820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59aa62a-2957-4466-822f-57fa185afa8c service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:07.014233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59aa62a-2957-4466-822f-57fa185afa8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:07.017233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c59aa62a-2957-4466-822f-57fa185afa8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:07.184421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59aa62a-2957-4466-822f-57fa185afa8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:07.186211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59aa62a-2957-4466-822f-57fa185afa8c service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:07.227485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 680/1400] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 785 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:10.479674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55979df7-0475-4061-be0a-d3babe9e08e5 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:10.480320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55979df7-0475-4061-be0a-d3babe9e08e5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:10.481214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55979df7-0475-4061-be0a-d3babe9e08e5 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:10.483700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 721/1401] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:55:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:10.492391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f255d4-5b09-4a99-9589-3b61965f0fa8 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:10.493813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f255d4-5b09-4a99-9589-3b61965f0fa8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:10.493813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f255d4-5b09-4a99-9589-3b61965f0fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:10.558384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:10.588859 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-98f255d4-5b09-4a99-9589-3b61965f0fa8 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:10.591881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-98f255d4-5b09-4a99-9589-3b61965f0fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:10.762353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 681/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:55:11.167982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55979df7-0475-4061-be0a-d3babe9e08e5 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:11.171526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-55979df7-0475-4061-be0a-d3babe9e08e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:11.319565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:11.487041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55979df7-0475-4061-be0a-d3babe9e08e5 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:11.634486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ae191-a2a4-48e6-abf7-bfcd26ea1321 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:11.635106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae191-a2a4-48e6-abf7-bfcd26ea1321 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:11.635384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae191-a2a4-48e6-abf7-bfcd26ea1321 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:11.635658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae191-a2a4-48e6-abf7-bfcd26ea1321 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:11.637396 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e54ae191-a2a4-48e6-abf7-bfcd26ea1321 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:11.668243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:11.765166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ae191-a2a4-48e6-abf7-bfcd26ea1321 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:11.766779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ae191-a2a4-48e6-abf7-bfcd26ea1321 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:11.793659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 722/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:14.413322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842815d9-af6b-46c1-b2c7-8bd8a0325780 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:14.414167 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842815d9-af6b-46c1-b2c7-8bd8a0325780 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:14.414167 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842815d9-af6b-46c1-b2c7-8bd8a0325780 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:14.415688 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 682/1404] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:14.424258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bcf69-dcd9-422e-ae23-c09bf985dc5a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:14.425077 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bcf69-dcd9-422e-ae23-c09bf985dc5a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:14.425077 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bcf69-dcd9-422e-ae23-c09bf985dc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:14.524953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2e3bcf69-dcd9-422e-ae23-c09bf985dc5a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:14.526703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2e3bcf69-dcd9-422e-ae23-c09bf985dc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:14.700797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:14.712376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 723/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:15.002558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842815d9-af6b-46c1-b2c7-8bd8a0325780 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:15.004130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-842815d9-af6b-46c1-b2c7-8bd8a0325780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:15.208636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:15.253055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842815d9-af6b-46c1-b2c7-8bd8a0325780 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:15.419288 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f88e3b-e0e4-4e28-86ed-0596740f940f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:15.420077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f88e3b-e0e4-4e28-86ed-0596740f940f service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:15.420327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f88e3b-e0e4-4e28-86ed-0596740f940f service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:15.420612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f88e3b-e0e4-4e28-86ed-0596740f940f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:15.422321 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-62f88e3b-e0e4-4e28-86ed-0596740f940f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:15.554538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:15.572007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f88e3b-e0e4-4e28-86ed-0596740f940f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:15.573699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f88e3b-e0e4-4e28-86ed-0596740f940f service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:15.618922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 683/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:55:18.299561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcdffc-ad52-4746-9a02-d95087d477e7 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:18.300529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcdffc-ad52-4746-9a02-d95087d477e7 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:18.300703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcdffc-ad52-4746-9a02-d95087d477e7 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:18.303515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 724/1407] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:55:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:18.313362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafc05f-b61a-406e-8f3c-60bf61774336 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:18.314190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafc05f-b61a-406e-8f3c-60bf61774336 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:18.314342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafc05f-b61a-406e-8f3c-60bf61774336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:18.524469 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-8eafc05f-b61a-406e-8f3c-60bf61774336 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:18.671293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8eafc05f-b61a-406e-8f3c-60bf61774336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:18.694616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 684/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:19.061365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcdffc-ad52-4746-9a02-d95087d477e7 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:19.062898 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-83dcdffc-ad52-4746-9a02-d95087d477e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:19.234352 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcdffc-ad52-4746-9a02-d95087d477e7 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:19.319794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:19.376470 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357efe8-d46b-4038-b1fb-4b2a851adcae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:19.377106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357efe8-d46b-4038-b1fb-4b2a851adcae service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:19.377387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357efe8-d46b-4038-b1fb-4b2a851adcae service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:19.377660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357efe8-d46b-4038-b1fb-4b2a851adcae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:19.379374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1357efe8-d46b-4038-b1fb-4b2a851adcae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:19.489843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357efe8-d46b-4038-b1fb-4b2a851adcae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:19.491532 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357efe8-d46b-4038-b1fb-4b2a851adcae service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:19.512138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 725/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:55:28.692741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=76 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:29.533755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=62 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:30.941519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f95f0-1aac-44dd-9fbb-a49130bc5065 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:30.942152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f95f0-1aac-44dd-9fbb-a49130bc5065 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:30.942460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f95f0-1aac-44dd-9fbb-a49130bc5065 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:30.944367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 685/1410] 10.222.0.15 () {54 vars in 918 bytes} [Tue Oct 4 11:55:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:30.950127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db968cc7-0955-4c06-9c5e-919ccc51804b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:30.950711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db968cc7-0955-4c06-9c5e-919ccc51804b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:30.950711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db968cc7-0955-4c06-9c5e-919ccc51804b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:31.036715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-db968cc7-0955-4c06-9c5e-919ccc51804b None None] MFA Rules not processed for user `16374dbb6d084ddfa3b49afd19c7c325`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:31.037929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-db968cc7-0955-4c06-9c5e-919ccc51804b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:31.156854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 726/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Tue Oct 4 11:55:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:31.173660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90491771-a8c4-4158-9cdb-8ccc472586b6 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:31.173929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90491771-a8c4-4158-9cdb-8ccc472586b6 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:31.174211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90491771-a8c4-4158-9cdb-8ccc472586b6 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:31.175832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 686/1412] 10.222.0.15 () {54 vars in 918 bytes} [Tue Oct 4 11:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:31.182324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6facdb7-a0e7-4fd9-81d5-c7257842dce6 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:31.182611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6facdb7-a0e7-4fd9-81d5-c7257842dce6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:31.182893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6facdb7-a0e7-4fd9-81d5-c7257842dce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:31.270294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:31.274676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e6facdb7-a0e7-4fd9-81d5-c7257842dce6 None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:31.275997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e6facdb7-a0e7-4fd9-81d5-c7257842dce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:31.406854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 727/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Tue Oct 4 11:55:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:34.134542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191ff88-0846-4a6a-8d90-524784993ef1 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:34.135152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191ff88-0846-4a6a-8d90-524784993ef1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:34.135152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191ff88-0846-4a6a-8d90-524784993ef1 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:34.136777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 687/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:34.143105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd1537-28ad-463e-b4f5-b04391d73f2f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:34.143650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd1537-28ad-463e-b4f5-b04391d73f2f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:34.143785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd1537-28ad-463e-b4f5-b04391d73f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:34.220859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f4dd1537-28ad-463e-b4f5-b04391d73f2f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:34.222143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f4dd1537-28ad-463e-b4f5-b04391d73f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:34.291808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:34.339606 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 728/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:34.637624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191ff88-0846-4a6a-8d90-524784993ef1 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:34.639185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2191ff88-0846-4a6a-8d90-524784993ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:34.828746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191ff88-0846-4a6a-8d90-524784993ef1 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:34.927910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:34.979293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296ba18-569d-4725-9f3a-89095e22943d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:34.980038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296ba18-569d-4725-9f3a-89095e22943d service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:34.980392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296ba18-569d-4725-9f3a-89095e22943d service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:34.980706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296ba18-569d-4725-9f3a-89095e22943d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:34.982550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a296ba18-569d-4725-9f3a-89095e22943d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:35.097752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296ba18-569d-4725-9f3a-89095e22943d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:35.099423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296ba18-569d-4725-9f3a-89095e22943d service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:35.122099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 688/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:36.337211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978396b-7371-4c3a-b2e8-363d1cf976d3 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:36.338094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978396b-7371-4c3a-b2e8-363d1cf976d3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:36.338094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978396b-7371-4c3a-b2e8-363d1cf976d3 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:36.339774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 729/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 11:55:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:36.346870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d408f5b-2ae1-4a23-9f01-c4d87a7b64b9 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:36.347798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d408f5b-2ae1-4a23-9f01-c4d87a7b64b9 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:36.347798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d408f5b-2ae1-4a23-9f01-c4d87a7b64b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:36.432618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-9d408f5b-2ae1-4a23-9f01-c4d87a7b64b9 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:36.433895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-9d408f5b-2ae1-4a23-9f01-c4d87a7b64b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:36.464387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:36.542326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 689/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:36.704822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978396b-7371-4c3a-b2e8-363d1cf976d3 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:36.706423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f978396b-7371-4c3a-b2e8-363d1cf976d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:36.869859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978396b-7371-4c3a-b2e8-363d1cf976d3 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:36.913406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:36.990113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a3e74-c23b-48c4-8881-3b3721c85044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:36.990634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a3e74-c23b-48c4-8881-3b3721c85044 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:36.990915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a3e74-c23b-48c4-8881-3b3721c85044 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:36.991199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a3e74-c23b-48c4-8881-3b3721c85044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:36.993410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7b1a3e74-c23b-48c4-8881-3b3721c85044 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:37.092741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a3e74-c23b-48c4-8881-3b3721c85044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:37.094391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a3e74-c23b-48c4-8881-3b3721c85044 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:37.115674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 730/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:38.262000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff9fc1-4cd1-4acb-adce-ad31c3cf279b None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:38.262606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff9fc1-4cd1-4acb-adce-ad31c3cf279b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:38.262945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff9fc1-4cd1-4acb-adce-ad31c3cf279b None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:38.265434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 690/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 11:55:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:38.272277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af89ce-d8ab-4d23-bbf5-60956ab1cc9e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:38.272847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af89ce-d8ab-4d23-bbf5-60956ab1cc9e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:38.272847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af89ce-d8ab-4d23-bbf5-60956ab1cc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:38.327811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:38.351083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-86af89ce-d8ab-4d23-bbf5-60956ab1cc9e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:38.352346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-86af89ce-d8ab-4d23-bbf5-60956ab1cc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:38.450611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 731/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 11:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:38.616525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff9fc1-4cd1-4acb-adce-ad31c3cf279b None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:38.618532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cfff9fc1-4cd1-4acb-adce-ad31c3cf279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:38.780430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff9fc1-4cd1-4acb-adce-ad31c3cf279b None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:38.794716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:38.903483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f27f89-fe60-44f1-a475-cc8e0c98619a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:38.903999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f27f89-fe60-44f1-a475-cc8e0c98619a service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:38.904308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f27f89-fe60-44f1-a475-cc8e0c98619a service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:38.904585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f27f89-fe60-44f1-a475-cc8e0c98619a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:38.906306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f7f27f89-fe60-44f1-a475-cc8e0c98619a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:39.016964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f27f89-fe60-44f1-a475-cc8e0c98619a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:39.018602 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f27f89-fe60-44f1-a475-cc8e0c98619a service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:39.039280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 691/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:40.232151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c038c-acc4-4ac7-9539-6fb1ec1dd6d9 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:40.232758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c038c-acc4-4ac7-9539-6fb1ec1dd6d9 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:40.232758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c038c-acc4-4ac7-9539-6fb1ec1dd6d9 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:40.234091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 732/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 11:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:40.243017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba6329-b428-499e-8423-cb5f532ba8ac None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:40.243550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba6329-b428-499e-8423-cb5f532ba8ac None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:40.243550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba6329-b428-499e-8423-cb5f532ba8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:40.264673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:40.348962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-68ba6329-b428-499e-8423-cb5f532ba8ac None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:40.350182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-68ba6329-b428-499e-8423-cb5f532ba8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:40.470180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 692/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 11:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:40.625988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c038c-acc4-4ac7-9539-6fb1ec1dd6d9 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:40.627644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1a9c038c-acc4-4ac7-9539-6fb1ec1dd6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:40.753166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:40.793738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c038c-acc4-4ac7-9539-6fb1ec1dd6d9 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:40.920200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78564dea-d9d1-42ae-9853-110600864081 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:40.920814 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78564dea-d9d1-42ae-9853-110600864081 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:40.921088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78564dea-d9d1-42ae-9853-110600864081 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:40.921354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78564dea-d9d1-42ae-9853-110600864081 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:40.923026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-78564dea-d9d1-42ae-9853-110600864081 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:41.007154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:41.024536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78564dea-d9d1-42ae-9853-110600864081 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:41.026197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78564dea-d9d1-42ae-9853-110600864081 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:41.044730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 733/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:42.228341 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0e55b-0a3a-413e-ac8e-98f89fc867e9 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:42.229052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0e55b-0a3a-413e-ac8e-98f89fc867e9 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:42.229052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0e55b-0a3a-413e-ac8e-98f89fc867e9 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:42.230277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 693/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 11:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:42.237503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb6a20-34b0-4598-b862-9827f9ded960 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:42.238071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb6a20-34b0-4598-b862-9827f9ded960 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:42.238071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb6a20-34b0-4598-b862-9827f9ded960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:42.323076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1abb6a20-34b0-4598-b862-9827f9ded960 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:42.324353 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1abb6a20-34b0-4598-b862-9827f9ded960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:42.433931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 734/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 11:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:42.589019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0e55b-0a3a-413e-ac8e-98f89fc867e9 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:42.590545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1ed0e55b-0a3a-413e-ac8e-98f89fc867e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:42.696312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:42.775558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0e55b-0a3a-413e-ac8e-98f89fc867e9 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:42.945359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177d470-8f76-43ed-bcf3-5d4b43437776 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:42.946011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177d470-8f76-43ed-bcf3-5d4b43437776 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:42.946388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177d470-8f76-43ed-bcf3-5d4b43437776 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:42.946861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177d470-8f76-43ed-bcf3-5d4b43437776 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:42.949831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4177d470-8f76-43ed-bcf3-5d4b43437776 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:43.021200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:43.071935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177d470-8f76-43ed-bcf3-5d4b43437776 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:43.073668 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177d470-8f76-43ed-bcf3-5d4b43437776 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:43.096597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 694/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:44.306425 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f512a-97e4-49d1-a33d-ba9cc8d54918 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:44.307311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f512a-97e4-49d1-a33d-ba9cc8d54918 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:44.307311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f512a-97e4-49d1-a33d-ba9cc8d54918 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:44.308867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 735/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 11:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:55:44.316261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98452776-1127-4420-927f-fd231c3ee41a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:44.316969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98452776-1127-4420-927f-fd231c3ee41a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:44.316969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98452776-1127-4420-927f-fd231c3ee41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:44.415375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-98452776-1127-4420-927f-fd231c3ee41a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:44.417045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-98452776-1127-4420-927f-fd231c3ee41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:44.528962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 695/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 11:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:55:44.828614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f512a-97e4-49d1-a33d-ba9cc8d54918 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:44.830225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-500f512a-97e4-49d1-a33d-ba9cc8d54918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:44.896193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:45.010675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f512a-97e4-49d1-a33d-ba9cc8d54918 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:45.137151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf94f86-6809-4d90-9599-b6bcbef3952b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:45.137828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94f86-6809-4d90-9599-b6bcbef3952b service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:45.138107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94f86-6809-4d90-9599-b6bcbef3952b service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:45.138376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf94f86-6809-4d90-9599-b6bcbef3952b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:45.140038 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8bf94f86-6809-4d90-9599-b6bcbef3952b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:45.148051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:45.241096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf94f86-6809-4d90-9599-b6bcbef3952b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:45.243548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf94f86-6809-4d90-9599-b6bcbef3952b service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:45.267963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 736/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:55:46.433293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade264-1dee-43ef-ae5e-0ae75ff6a0d5 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:46.433869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade264-1dee-43ef-ae5e-0ae75ff6a0d5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:46.433987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade264-1dee-43ef-ae5e-0ae75ff6a0d5 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:46.436231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 696/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 11:55:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:55:46.443109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446ea1d-7ed8-4f8a-a350-2eea83ab6584 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:46.443710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446ea1d-7ed8-4f8a-a350-2eea83ab6584 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:46.443710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446ea1d-7ed8-4f8a-a350-2eea83ab6584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:46.531779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b446ea1d-7ed8-4f8a-a350-2eea83ab6584 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:46.533149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b446ea1d-7ed8-4f8a-a350-2eea83ab6584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:46.614870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:46.656455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 737/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 11:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:46.817461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ade264-1dee-43ef-ae5e-0ae75ff6a0d5 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:46.820850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-57ade264-1dee-43ef-ae5e-0ae75ff6a0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:46.843311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:47.031207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ade264-1dee-43ef-ae5e-0ae75ff6a0d5 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:47.139853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:47.170933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a926c7-f3f1-4a6b-8feb-48ce3f91ce49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:47.171548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a926c7-f3f1-4a6b-8feb-48ce3f91ce49 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:47.171879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a926c7-f3f1-4a6b-8feb-48ce3f91ce49 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:47.172252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a926c7-f3f1-4a6b-8feb-48ce3f91ce49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:47.173945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-77a926c7-f3f1-4a6b-8feb-48ce3f91ce49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:47.282155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a926c7-f3f1-4a6b-8feb-48ce3f91ce49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:47.283816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a926c7-f3f1-4a6b-8feb-48ce3f91ce49 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:47.303903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 697/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:48.523639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f7ed-7218-4515-874a-1263950fceda None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:48.524285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f7ed-7218-4515-874a-1263950fceda None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:48.524285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f7ed-7218-4515-874a-1263950fceda None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:48.525762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 738/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 11:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:48.533432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad49d9-846c-444b-84a9-a2652a1d8988 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:48.533953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad49d9-846c-444b-84a9-a2652a1d8988 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:48.534089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad49d9-846c-444b-84a9-a2652a1d8988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:48.621010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e6ad49d9-846c-444b-84a9-a2652a1d8988 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:48.622300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e6ad49d9-846c-444b-84a9-a2652a1d8988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:48.669649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:48.726803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 698/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 11:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:48.893737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574f7ed-7218-4515-874a-1263950fceda None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:48.895361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6574f7ed-7218-4515-874a-1263950fceda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:49.065000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574f7ed-7218-4515-874a-1263950fceda None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:49.131120 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:49.194175 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b94aa-6570-4bd3-9560-0b0ae61278e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:49.195797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b94aa-6570-4bd3-9560-0b0ae61278e3 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:49.196545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b94aa-6570-4bd3-9560-0b0ae61278e3 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:49.197338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b94aa-6570-4bd3-9560-0b0ae61278e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:49.199322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-290b94aa-6570-4bd3-9560-0b0ae61278e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:49.300676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b94aa-6570-4bd3-9560-0b0ae61278e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:49.302468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b94aa-6570-4bd3-9560-0b0ae61278e3 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:49.322852 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 739/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:50.581656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e36f80-e3e0-4f72-973d-9fbf5d2aafdb None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:50.582339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e36f80-e3e0-4f72-973d-9fbf5d2aafdb None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:50.582339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e36f80-e3e0-4f72-973d-9fbf5d2aafdb None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:50.583692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 699/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 11:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:50.591617 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d18c-b563-4505-b290-11a3b20215df None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:50.592292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d18c-b563-4505-b290-11a3b20215df None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:50.592292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d18c-b563-4505-b290-11a3b20215df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:50.696454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d244d18c-b563-4505-b290-11a3b20215df None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:50.697796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d244d18c-b563-4505-b290-11a3b20215df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:50.700501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:50.824257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 740/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 11:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:50.988704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e36f80-e3e0-4f72-973d-9fbf5d2aafdb None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:50.990463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a1e36f80-e3e0-4f72-973d-9fbf5d2aafdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:51.193103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e36f80-e3e0-4f72-973d-9fbf5d2aafdb None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:51.228128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:51.337066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864309f-b00a-429e-94e3-d38f49f5f930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:51.337598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864309f-b00a-429e-94e3-d38f49f5f930 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:51.337884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864309f-b00a-429e-94e3-d38f49f5f930 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:51.338156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864309f-b00a-429e-94e3-d38f49f5f930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:51.339864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b864309f-b00a-429e-94e3-d38f49f5f930 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:51.457169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864309f-b00a-429e-94e3-d38f49f5f930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:51.458796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864309f-b00a-429e-94e3-d38f49f5f930 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:51.484131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 700/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:53.727959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cbf35-236d-4554-b89b-990e61996ff6 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:53.728776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cbf35-236d-4554-b89b-990e61996ff6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:53.728776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cbf35-236d-4554-b89b-990e61996ff6 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:53.729951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 741/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:53.737325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b08a16-1b41-41f9-aa87-bf867e3575af None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:53.738128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b08a16-1b41-41f9-aa87-bf867e3575af None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:53.738297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b08a16-1b41-41f9-aa87-bf867e3575af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:53.818350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:53.866280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d2b08a16-1b41-41f9-aa87-bf867e3575af None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:53.868273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d2b08a16-1b41-41f9-aa87-bf867e3575af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:54.041492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 701/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:55:54.339716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cbf35-236d-4554-b89b-990e61996ff6 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:54.341543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-622cbf35-236d-4554-b89b-990e61996ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:54.516312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cbf35-236d-4554-b89b-990e61996ff6 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:54.562040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:54.608275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baf3ca-fc99-4b55-94e1-9ada4b6939c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:54.608897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf3ca-fc99-4b55-94e1-9ada4b6939c0 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:54.609178 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf3ca-fc99-4b55-94e1-9ada4b6939c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:54.609469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baf3ca-fc99-4b55-94e1-9ada4b6939c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:54.611360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-66baf3ca-fc99-4b55-94e1-9ada4b6939c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:54.748704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baf3ca-fc99-4b55-94e1-9ada4b6939c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:54.750799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baf3ca-fc99-4b55-94e1-9ada4b6939c0 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:54.771225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 742/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:55:57.500275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3848c7-3cd4-416d-9efd-e034f77a53b8 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:57.501133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3848c7-3cd4-416d-9efd-e034f77a53b8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:57.501133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3848c7-3cd4-416d-9efd-e034f77a53b8 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:57.502444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 702/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:55:57.511046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16927f-3d13-4d76-b041-a4961c0b3d5a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:57.511647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16927f-3d13-4d76-b041-a4961c0b3d5a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:57.511884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16927f-3d13-4d76-b041-a4961c0b3d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:57.598154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-7d16927f-3d13-4d76-b041-a4961c0b3d5a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:55:57.599545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7d16927f-3d13-4d76-b041-a4961c0b3d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:57.617363 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:57.792922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 743/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:55:58.087100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3848c7-3cd4-416d-9efd-e034f77a53b8 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:55:58.088893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-fd3848c7-3cd4-416d-9efd-e034f77a53b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:58.256005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3848c7-3cd4-416d-9efd-e034f77a53b8 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:55:58.276964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:55:58.352712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5d8f4-3987-4d6d-84a9-5a5bc9889796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:55:58.353316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5d8f4-3987-4d6d-84a9-5a5bc9889796 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:55:58.353597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5d8f4-3987-4d6d-84a9-5a5bc9889796 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:55:58.353863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5d8f4-3987-4d6d-84a9-5a5bc9889796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:55:58.355575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-97d5d8f4-3987-4d6d-84a9-5a5bc9889796 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:55:58.479312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5d8f4-3987-4d6d-84a9-5a5bc9889796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:55:58.481420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5d8f4-3987-4d6d-84a9-5a5bc9889796 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:55:58.503493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 703/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:56:01.456457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b0266-603f-4dcc-84d2-d0dc958807ff None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:01.457065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b0266-603f-4dcc-84d2-d0dc958807ff None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:01.457065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b0266-603f-4dcc-84d2-d0dc958807ff None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:01.458465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 744/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:56:01.464686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d20c9-52fb-4107-b088-e70a3c4630d5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:01.465555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d20c9-52fb-4107-b088-e70a3c4630d5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:01.465734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d20c9-52fb-4107-b088-e70a3c4630d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:01.525092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:01.548843 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-c49d20c9-52fb-4107-b088-e70a3c4630d5 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:01.550189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c49d20c9-52fb-4107-b088-e70a3c4630d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:01.670411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 704/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:01.979589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b0266-603f-4dcc-84d2-d0dc958807ff None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:01.981194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5c1b0266-603f-4dcc-84d2-d0dc958807ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:02.105666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b0266-603f-4dcc-84d2-d0dc958807ff None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:02.154642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:02.193568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74004eb7-357c-439c-9ced-cdd91eaae3ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:02.194237 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74004eb7-357c-439c-9ced-cdd91eaae3ef service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:02.194514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74004eb7-357c-439c-9ced-cdd91eaae3ef service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:02.194784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74004eb7-357c-439c-9ced-cdd91eaae3ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:02.196539 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-74004eb7-357c-439c-9ced-cdd91eaae3ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:02.344881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74004eb7-357c-439c-9ced-cdd91eaae3ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:02.346334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74004eb7-357c-439c-9ced-cdd91eaae3ef service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:02.366104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 745/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:56:04.998756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a30cd-5264-43ba-9292-5773d3438a12 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:04.999354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a30cd-5264-43ba-9292-5773d3438a12 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:04.999354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a30cd-5264-43ba-9292-5773d3438a12 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:05.000940 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 705/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:56:05.007247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1800-e70b-46b4-ba97-69f85a6b1f77 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:05.007824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1800-e70b-46b4-ba97-69f85a6b1f77 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:05.008097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1800-e70b-46b4-ba97-69f85a6b1f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:05.093703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-01ea1800-e70b-46b4-ba97-69f85a6b1f77 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:05.095090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-01ea1800-e70b-46b4-ba97-69f85a6b1f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:05.118659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:05.216666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 746/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:56:05.508755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a30cd-5264-43ba-9292-5773d3438a12 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:05.510579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-868a30cd-5264-43ba-9292-5773d3438a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:05.658129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a30cd-5264-43ba-9292-5773d3438a12 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:05.689686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:05.776953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c2571-c127-4ae6-95e7-5a36fce1e41e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:05.777754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c2571-c127-4ae6-95e7-5a36fce1e41e service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:05.778181 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c2571-c127-4ae6-95e7-5a36fce1e41e service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:05.778523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c2571-c127-4ae6-95e7-5a36fce1e41e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:05.780691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e02c2571-c127-4ae6-95e7-5a36fce1e41e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:05.918732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c2571-c127-4ae6-95e7-5a36fce1e41e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:05.920871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c2571-c127-4ae6-95e7-5a36fce1e41e service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:05.945834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 706/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:56:08.391149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7ceec-2051-4ad2-b2ab-b569746118a3 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:08.391790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7ceec-2051-4ad2-b2ab-b569746118a3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:08.391790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7ceec-2051-4ad2-b2ab-b569746118a3 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:08.393204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 747/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:56:08.400201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460602-20ac-4087-96d1-750d0bdc4f23 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:08.400922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460602-20ac-4087-96d1-750d0bdc4f23 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:08.400922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460602-20ac-4087-96d1-750d0bdc4f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:08.476557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:08.480108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-11460602-20ac-4087-96d1-750d0bdc4f23 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:08.481673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-11460602-20ac-4087-96d1-750d0bdc4f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:08.597579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 707/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:08.895921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7ceec-2051-4ad2-b2ab-b569746118a3 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:08.897567 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2cc7ceec-2051-4ad2-b2ab-b569746118a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:09.146704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7ceec-2051-4ad2-b2ab-b569746118a3 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:09.209286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:09.237355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43811526-580e-4d0d-a203-b92779fa44a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:09.237965 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43811526-580e-4d0d-a203-b92779fa44a6 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:09.238235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43811526-580e-4d0d-a203-b92779fa44a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:09.238504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43811526-580e-4d0d-a203-b92779fa44a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:09.240142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-43811526-580e-4d0d-a203-b92779fa44a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:09.369097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43811526-580e-4d0d-a203-b92779fa44a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:09.370607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43811526-580e-4d0d-a203-b92779fa44a6 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:09.392072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 748/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:56:11.841927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dab354-39e6-413e-a9d4-dc88f5bcfafe None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:11.842686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dab354-39e6-413e-a9d4-dc88f5bcfafe None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:11.842686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dab354-39e6-413e-a9d4-dc88f5bcfafe None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:11.843804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 708/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:56:11.851747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439fc2b-e103-4329-9125-1fd951774b2c None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:11.852395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439fc2b-e103-4329-9125-1fd951774b2c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:11.852395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439fc2b-e103-4329-9125-1fd951774b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:11.932633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-0439fc2b-e103-4329-9125-1fd951774b2c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:11.933868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-0439fc2b-e103-4329-9125-1fd951774b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:11.970351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:12.052663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 749/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:12.351977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dab354-39e6-413e-a9d4-dc88f5bcfafe None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:12.353655 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-92dab354-39e6-413e-a9d4-dc88f5bcfafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:12.491714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dab354-39e6-413e-a9d4-dc88f5bcfafe None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:12.526042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:12.582256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1947b-140d-4f82-8eab-15b006070e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:12.583178 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1947b-140d-4f82-8eab-15b006070e0c service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:12.583460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1947b-140d-4f82-8eab-15b006070e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:12.583775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1947b-140d-4f82-8eab-15b006070e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:12.586795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5ec1947b-140d-4f82-8eab-15b006070e0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:12.721694 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1947b-140d-4f82-8eab-15b006070e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:12.723082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1947b-140d-4f82-8eab-15b006070e0c service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:12.746040 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 709/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:56:15.318257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013de28d-152c-426c-9e73-e4d6870a343f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:15.319138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013de28d-152c-426c-9e73-e4d6870a343f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:15.319138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013de28d-152c-426c-9e73-e4d6870a343f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:15.320248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 750/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:56:15.326710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5563e-73ef-4a00-9e41-7070fd03079c None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:15.327319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5563e-73ef-4a00-9e41-7070fd03079c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:15.327319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5563e-73ef-4a00-9e41-7070fd03079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:15.411910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b2c5563e-73ef-4a00-9e41-7070fd03079c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:15.413714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b2c5563e-73ef-4a00-9e41-7070fd03079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:15.425071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:15.532908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 710/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:56:15.841486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013de28d-152c-426c-9e73-e4d6870a343f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:15.843193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-013de28d-152c-426c-9e73-e4d6870a343f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:15.978980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013de28d-152c-426c-9e73-e4d6870a343f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:16.053031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:16.080923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451f40c-e33d-474b-b322-033547e1a4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:16.081547 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451f40c-e33d-474b-b322-033547e1a4dd service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:16.081936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451f40c-e33d-474b-b322-033547e1a4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:16.082252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451f40c-e33d-474b-b322-033547e1a4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:16.084034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-0451f40c-e33d-474b-b322-033547e1a4dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:16.235448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451f40c-e33d-474b-b322-033547e1a4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:16.237156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451f40c-e33d-474b-b322-033547e1a4dd service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:16.262415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 751/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:56:18.581820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5797eba-c2bc-4528-ba68-105d94922b49 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:18.582445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5797eba-c2bc-4528-ba68-105d94922b49 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:18.582445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5797eba-c2bc-4528-ba68-105d94922b49 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:18.583827 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 711/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:56:18.593524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694269-bc95-49fe-88cd-85527a1baa30 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:18.594122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694269-bc95-49fe-88cd-85527a1baa30 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:18.594377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694269-bc95-49fe-88cd-85527a1baa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:18.691774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-a5694269-bc95-49fe-88cd-85527a1baa30 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:18.693902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a5694269-bc95-49fe-88cd-85527a1baa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:18.750977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:18.836705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 752/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:19.143715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5797eba-c2bc-4528-ba68-105d94922b49 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:19.145357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b5797eba-c2bc-4528-ba68-105d94922b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:19.287729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5797eba-c2bc-4528-ba68-105d94922b49 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:19.334455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:19.377130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd29ec-165b-4731-a127-6fb6935a83f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:19.377796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd29ec-165b-4731-a127-6fb6935a83f1 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:19.378103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd29ec-165b-4731-a127-6fb6935a83f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:19.378400 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd29ec-165b-4731-a127-6fb6935a83f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:19.380226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b3dd29ec-165b-4731-a127-6fb6935a83f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:19.514686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd29ec-165b-4731-a127-6fb6935a83f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:19.516050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd29ec-165b-4731-a127-6fb6935a83f1 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:19.539935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 712/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:56:22.140885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef71cb6-9cb1-4cc7-9993-0dddd61f3710 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:22.140885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef71cb6-9cb1-4cc7-9993-0dddd61f3710 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:22.140885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef71cb6-9cb1-4cc7-9993-0dddd61f3710 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:22.140885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 753/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:56:22.147280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d5062-e904-4f34-a8af-c0372b96d2a7 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:22.150804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d5062-e904-4f34-a8af-c0372b96d2a7 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:22.150804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d5062-e904-4f34-a8af-c0372b96d2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:22.227750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-501d5062-e904-4f34-a8af-c0372b96d2a7 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:22.229049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-501d5062-e904-4f34-a8af-c0372b96d2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:22.249173 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:22.340496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 713/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:22.661563 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef71cb6-9cb1-4cc7-9993-0dddd61f3710 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:22.663288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9ef71cb6-9cb1-4cc7-9993-0dddd61f3710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:22.811674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef71cb6-9cb1-4cc7-9993-0dddd61f3710 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:22.891955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:22.900384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb53779-db61-471d-96d9-3ca476e52075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:22.901074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb53779-db61-471d-96d9-3ca476e52075 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:22.901369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb53779-db61-471d-96d9-3ca476e52075 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:22.901672 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb53779-db61-471d-96d9-3ca476e52075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:22.903477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1bb53779-db61-471d-96d9-3ca476e52075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:23.037643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb53779-db61-471d-96d9-3ca476e52075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:23.039127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb53779-db61-471d-96d9-3ca476e52075 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:23.072395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 754/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:56:25.422404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee2ab7-b59e-4c32-88cf-73771e7acab2 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:25.423015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee2ab7-b59e-4c32-88cf-73771e7acab2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:25.423015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee2ab7-b59e-4c32-88cf-73771e7acab2 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:25.424329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 714/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 11:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:56:25.431898 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6688a7-7d91-4aaf-a52d-ca347c4031f6 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:25.432586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6688a7-7d91-4aaf-a52d-ca347c4031f6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:25.432586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6688a7-7d91-4aaf-a52d-ca347c4031f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:25.510558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-ec6688a7-7d91-4aaf-a52d-ca347c4031f6 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:25.511802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ec6688a7-7d91-4aaf-a52d-ca347c4031f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:25.570217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:25.627519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 755/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 11:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:25.918482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee2ab7-b59e-4c32-88cf-73771e7acab2 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:25.921375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-35ee2ab7-b59e-4c32-88cf-73771e7acab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:26.056821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee2ab7-b59e-4c32-88cf-73771e7acab2 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:26.118991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:26.150365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce841500-7143-49bb-b8a5-0dbaa2b7f8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:26.150974 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce841500-7143-49bb-b8a5-0dbaa2b7f8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:26.151248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce841500-7143-49bb-b8a5-0dbaa2b7f8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:26.151516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce841500-7143-49bb-b8a5-0dbaa2b7f8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:26.153324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ce841500-7143-49bb-b8a5-0dbaa2b7f8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:26.278579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce841500-7143-49bb-b8a5-0dbaa2b7f8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:26.280055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce841500-7143-49bb-b8a5-0dbaa2b7f8e9 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:26.302134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 715/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:56:35.625437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=22 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:36.299382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=66 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:47.537174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efe56b-8979-4d21-bd9c-f10117f3a81f None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:47.537864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efe56b-8979-4d21-bd9c-f10117f3a81f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:47.537864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efe56b-8979-4d21-bd9c-f10117f3a81f None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:47.539097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 756/1471] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:56:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:56:47.545222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09706c-320c-46f7-936f-ca7ea4d6acc4 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:47.545815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09706c-320c-46f7-936f-ca7ea4d6acc4 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:47.545815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09706c-320c-46f7-936f-ca7ea4d6acc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:47.622589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-4c09706c-320c-46f7-936f-ca7ea4d6acc4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:47.623941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4c09706c-320c-46f7-936f-ca7ea4d6acc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:47.741995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 716/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:48.053816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc0378-9fb8-427f-8b31-8a85785c5329 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:48.054007 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc0378-9fb8-427f-8b31-8a85785c5329 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:48.054312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc0378-9fb8-427f-8b31-8a85785c5329 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:48.055831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 757/1473] 10.222.0.15 () {54 vars in 962 bytes} [Tue Oct 4 11:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:56:48.061164 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cab064-e220-43f1-9e92-7f8c6635c41e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:48.061799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cab064-e220-43f1-9e92-7f8c6635c41e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:48.062143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cab064-e220-43f1-9e92-7f8c6635c41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:48.156566 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f9cab064-e220-43f1-9e92-7f8c6635c41e None None] MFA Rules not processed for user `454fb811f6e34e1d881d4948ede058b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:48.157813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f9cab064-e220-43f1-9e92-7f8c6635c41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:48.182838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:48.261822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 717/1474] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Oct 4 11:56:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:48.270926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc0378-9fb8-427f-8b31-8a85785c5329 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:48.273899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8cbc0378-9fb8-427f-8b31-8a85785c5329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:48.399319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc0378-9fb8-427f-8b31-8a85785c5329 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:48.481189 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24344f-24d2-4e25-88a8-aeba05516085 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454fb811f6e34e1d881d4948ede058b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:48.482022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24344f-24d2-4e25-88a8-aeba05516085 service glance] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:48.482307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24344f-24d2-4e25-88a8-aeba05516085 service glance] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:48.482580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24344f-24d2-4e25-88a8-aeba05516085 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:48.485188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-0f24344f-24d2-4e25-88a8-aeba05516085 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:48.549331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:48.615311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24344f-24d2-4e25-88a8-aeba05516085 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:48.616903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24344f-24d2-4e25-88a8-aeba05516085 service glance] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:48.638819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 758/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:56:51.064759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283271c-2558-41d6-81cb-1963250b7cd5 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:51.065365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283271c-2558-41d6-81cb-1963250b7cd5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:51.065617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283271c-2558-41d6-81cb-1963250b7cd5 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:51.068152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 718/1476] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:56:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:56:51.074721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65a9c3-0b0c-4ce6-9eef-769b98afaf28 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:51.075667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65a9c3-0b0c-4ce6-9eef-769b98afaf28 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:51.075667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65a9c3-0b0c-4ce6-9eef-769b98afaf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:51.160626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-6c65a9c3-0b0c-4ce6-9eef-769b98afaf28 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:51.161991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6c65a9c3-0b0c-4ce6-9eef-769b98afaf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:51.286605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 759/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:51.579709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283271c-2558-41d6-81cb-1963250b7cd5 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:51.581762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1283271c-2558-41d6-81cb-1963250b7cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:51.759183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283271c-2558-41d6-81cb-1963250b7cd5 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:51.819631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:51.868593 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff0bd6-c9d8-464c-b41a-2e34fa134020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:51.869217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0bd6-c9d8-464c-b41a-2e34fa134020 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:51.869756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0bd6-c9d8-464c-b41a-2e34fa134020 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:51.869756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0bd6-c9d8-464c-b41a-2e34fa134020 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:51.871140 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff0bd6-c9d8-464c-b41a-2e34fa134020 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:51.872420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WARNING py.warnings [None req-77ff0bd6-c9d8-464c-b41a-2e34fa134020 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 11:56:51.872420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: warnings.warn(msg) Oct 04 11:56:51.872420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: Oct 04 11:56:51.873571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff0bd6-c9d8-464c-b41a-2e34fa134020 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:51.897173 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 719/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Oct 4 11:56:51 2022] GET /identity/v3/projects => generated 2553 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 11:56:54.194724 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf5740-419d-4fbb-a2ef-dee8e858c1be None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:54.195406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf5740-419d-4fbb-a2ef-dee8e858c1be None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:54.195406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf5740-419d-4fbb-a2ef-dee8e858c1be None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:54.196879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 760/1479] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:56:54.203468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7837af-77cc-4ac8-85ae-287985c44674 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:54.204195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7837af-77cc-4ac8-85ae-287985c44674 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:54.204195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7837af-77cc-4ac8-85ae-287985c44674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:54.283920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-0b7837af-77cc-4ac8-85ae-287985c44674 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:54.285224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0b7837af-77cc-4ac8-85ae-287985c44674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:54.527236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 720/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:54.834411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf5740-419d-4fbb-a2ef-dee8e858c1be None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:54.841862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-30cf5740-419d-4fbb-a2ef-dee8e858c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:54.847477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:55.021770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf5740-419d-4fbb-a2ef-dee8e858c1be None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:55.120309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:55.159885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bb4d7-c287-4595-9886-0194de4dea0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:55.160527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bb4d7-c287-4595-9886-0194de4dea0b service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:55.161334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bb4d7-c287-4595-9886-0194de4dea0b service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:55.161608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bb4d7-c287-4595-9886-0194de4dea0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:55.163302 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-278bb4d7-c287-4595-9886-0194de4dea0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:55.276101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bb4d7-c287-4595-9886-0194de4dea0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:55.277909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bb4d7-c287-4595-9886-0194de4dea0b service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:55.298465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 761/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:56:57.777086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd68db0-76d0-48da-b1ea-ebab543c834d None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:57.777689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd68db0-76d0-48da-b1ea-ebab543c834d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:57.777689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd68db0-76d0-48da-b1ea-ebab543c834d None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:57.779016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 721/1482] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:56:57.785157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62caadd-84d4-4801-a379-d8e189c64757 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:57.785737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62caadd-84d4-4801-a379-d8e189c64757 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:57.785737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62caadd-84d4-4801-a379-d8e189c64757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:57.877964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d62caadd-84d4-4801-a379-d8e189c64757 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:56:57.880074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d62caadd-84d4-4801-a379-d8e189c64757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:57.970753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:58.037804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 762/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:56:58.338246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd68db0-76d0-48da-b1ea-ebab543c834d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:56:58.341072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3dd68db0-76d0-48da-b1ea-ebab543c834d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:58.385542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:58.540528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd68db0-76d0-48da-b1ea-ebab543c834d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:56:58.645725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe43ed-c9cf-4a4d-a5d5-ba03646aead5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:56:58.646579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe43ed-c9cf-4a4d-a5d5-ba03646aead5 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:56:58.646923 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe43ed-c9cf-4a4d-a5d5-ba03646aead5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:56:58.647340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe43ed-c9cf-4a4d-a5d5-ba03646aead5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:56:58.649252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8ffe43ed-c9cf-4a4d-a5d5-ba03646aead5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:56:58.755504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:56:58.795935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe43ed-c9cf-4a4d-a5d5-ba03646aead5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:56:58.797565 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe43ed-c9cf-4a4d-a5d5-ba03646aead5 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:56:58.819405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 722/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:57:01.197742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c5cbd-ef70-40e3-8f88-6f4f3bd76464 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:57:01.198351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c5cbd-ef70-40e3-8f88-6f4f3bd76464 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:57:01.198351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c5cbd-ef70-40e3-8f88-6f4f3bd76464 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:57:01.199628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 763/1485] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:57:01.207073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419ad6-9adf-45ab-9df7-d9343d047a16 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:57:01.208081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419ad6-9adf-45ab-9df7-d9343d047a16 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:57:01.208081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419ad6-9adf-45ab-9df7-d9343d047a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:57:01.281203 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-da419ad6-9adf-45ab-9df7-d9343d047a16 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:57:01.282438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-da419ad6-9adf-45ab-9df7-d9343d047a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:57:01.394266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 723/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:57:01.688114 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c5cbd-ef70-40e3-8f88-6f4f3bd76464 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:57:01.689719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5d1c5cbd-ef70-40e3-8f88-6f4f3bd76464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:57:01.820728 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c5cbd-ef70-40e3-8f88-6f4f3bd76464 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:57:01.905711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:57:01.907402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169c959-db84-4a60-aef3-a3126a53306b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:57:01.908024 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c959-db84-4a60-aef3-a3126a53306b service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:57:01.908336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c959-db84-4a60-aef3-a3126a53306b service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:57:01.908611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169c959-db84-4a60-aef3-a3126a53306b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:57:01.910416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7169c959-db84-4a60-aef3-a3126a53306b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:57:02.043821 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169c959-db84-4a60-aef3-a3126a53306b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:57:02.045377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169c959-db84-4a60-aef3-a3126a53306b service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:57:02.066531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 764/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:57:04.457536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7f0d9-6863-44cd-b1c3-d4817e6c78e8 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:57:04.458273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7f0d9-6863-44cd-b1c3-d4817e6c78e8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:57:04.458273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7f0d9-6863-44cd-b1c3-d4817e6c78e8 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:57:04.459469 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 724/1488] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:57:04.465862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d32f1f-a603-433e-a794-1197551a0992 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:57:04.466495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d32f1f-a603-433e-a794-1197551a0992 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:57:04.466495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d32f1f-a603-433e-a794-1197551a0992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:57:04.547873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-53d32f1f-a603-433e-a794-1197551a0992 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:57:04.549186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-53d32f1f-a603-433e-a794-1197551a0992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:57:04.613107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:57:04.688800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 765/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:57:04.999268 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7f0d9-6863-44cd-b1c3-d4817e6c78e8 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:57:05.001368 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-71d7f0d9-6863-44cd-b1c3-d4817e6c78e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:57:05.043228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:57:05.123887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7f0d9-6863-44cd-b1c3-d4817e6c78e8 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:57:05.212535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89c44a-3c58-4ab2-9cea-6b347bb9399d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:57:05.213163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89c44a-3c58-4ab2-9cea-6b347bb9399d service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:57:05.213435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89c44a-3c58-4ab2-9cea-6b347bb9399d service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:57:05.213709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89c44a-3c58-4ab2-9cea-6b347bb9399d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:57:05.215423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8e89c44a-3c58-4ab2-9cea-6b347bb9399d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:57:05.309158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:57:05.351132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89c44a-3c58-4ab2-9cea-6b347bb9399d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:57:05.352817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89c44a-3c58-4ab2-9cea-6b347bb9399d service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:57:05.373107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 725/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:57:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 11:57:08.084488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ff1c9-2bc4-4542-a28d-32e1698289f1 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:57:08.085070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ff1c9-2bc4-4542-a28d-32e1698289f1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:57:08.085340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ff1c9-2bc4-4542-a28d-32e1698289f1 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:57:08.087136 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 766/1491] 10.222.0.15 () {54 vars in 927 bytes} [Tue Oct 4 11:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:57:08.093296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522df143-68f6-4964-8a5e-0f50bcb41b2b None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:57:08.093805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522df143-68f6-4964-8a5e-0f50bcb41b2b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:57:08.093895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522df143-68f6-4964-8a5e-0f50bcb41b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:57:08.169684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-522df143-68f6-4964-8a5e-0f50bcb41b2b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:57:08.170910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-522df143-68f6-4964-8a5e-0f50bcb41b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:57:08.278263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 726/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Oct 4 11:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:57:08.573061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ff1c9-2bc4-4542-a28d-32e1698289f1 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:57:08.574617 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2f9ff1c9-2bc4-4542-a28d-32e1698289f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:57:08.692473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ff1c9-2bc4-4542-a28d-32e1698289f1 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:57:08.790916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64329b-10a0-4cab-abb8-b16ea6afccf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:57:08.791583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64329b-10a0-4cab-abb8-b16ea6afccf7 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:57:08.791912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64329b-10a0-4cab-abb8-b16ea6afccf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:57:08.792513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64329b-10a0-4cab-abb8-b16ea6afccf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:57:08.794622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8f64329b-10a0-4cab-abb8-b16ea6afccf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:57:08.808498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:57:08.933239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64329b-10a0-4cab-abb8-b16ea6afccf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:57:08.934736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64329b-10a0-4cab-abb8-b16ea6afccf7 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:57:08.956153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 767/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:57:18.276524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=82 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:57:18.953411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=50 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:58:40.432424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe94f61-bc68-462d-8f48-7b5897907efb None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:40.433379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe94f61-bc68-462d-8f48-7b5897907efb None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:40.433379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe94f61-bc68-462d-8f48-7b5897907efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:40.537866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-efe94f61-bc68-462d-8f48-7b5897907efb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:58:40.539104 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-efe94f61-bc68-462d-8f48-7b5897907efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:58:40.664485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 727/1494] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 11:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:58:40.673379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13d1aa-ac10-4836-ab0c-235fc7491c85 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:40.674182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13d1aa-ac10-4836-ab0c-235fc7491c85 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:40.674182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13d1aa-ac10-4836-ab0c-235fc7491c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:40.699500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-3a13d1aa-ac10-4836-ab0c-235fc7491c85 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:58:40.700806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3a13d1aa-ac10-4836-ab0c-235fc7491c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:58:40.710760 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 768/1495] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 11:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:58:40.715741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe94f61-bc68-462d-8f48-7b5897907efb None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:58:40.726679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe94f61-bc68-462d-8f48-7b5897907efb None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:58:40.733926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dada8-f23b-4686-9b95-4ab30515f802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:58:40.734563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dada8-f23b-4686-9b95-4ab30515f802 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:40.734842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dada8-f23b-4686-9b95-4ab30515f802 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:40.735128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163dada8-f23b-4686-9b95-4ab30515f802 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:40.736741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163dada8-f23b-4686-9b95-4ab30515f802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 11:58:40.737034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163dada8-f23b-4686-9b95-4ab30515f802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:58:40.738103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WARNING py.warnings [None req-163dada8-f23b-4686-9b95-4ab30515f802 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 11:58:40.738103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: warnings.warn(msg) Oct 04 11:58:40.738103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: Oct 04 11:58:40.739955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163dada8-f23b-4686-9b95-4ab30515f802 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:58:40.754608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 728/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 11:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 11:58:40.763692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13d1aa-ac10-4836-ab0c-235fc7491c85 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:58:40.773947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13d1aa-ac10-4836-ab0c-235fc7491c85 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:58:40.779090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e1c7d-39e5-40c9-99b4-3e822ecbc87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:58:40.779715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e1c7d-39e5-40c9-99b4-3e822ecbc87f admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:40.779994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e1c7d-39e5-40c9-99b4-3e822ecbc87f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:40.780303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e1c7d-39e5-40c9-99b4-3e822ecbc87f admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:40.781032 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e1c7d-39e5-40c9-99b4-3e822ecbc87f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:58:40.782999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: WARNING py.warnings [None req-190e1c7d-39e5-40c9-99b4-3e822ecbc87f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 11:58:40.782999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: warnings.warn(msg) Oct 04 11:58:40.782999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: Oct 04 11:58:40.784262 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e1c7d-39e5-40c9-99b4-3e822ecbc87f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:58:40.985543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 769/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 11:58:40 2022] POST /identity/v3/projects => generated 377 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:58:40.990092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dada8-f23b-4686-9b95-4ab30515f802 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:58:41.001376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163dada8-f23b-4686-9b95-4ab30515f802 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:58:41.095851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:58:41.107998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fa917-ae81-4700-bd48-c7a370a2a2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:58:41.109108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fa917-ae81-4700-bd48-c7a370a2a2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:41.109609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fa917-ae81-4700-bd48-c7a370a2a2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:41.110691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fa917-ae81-4700-bd48-c7a370a2a2f0 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:41.111904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641fa917-ae81-4700-bd48-c7a370a2a2f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:58:41.113847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WARNING py.warnings [None req-641fa917-ae81-4700-bd48-c7a370a2a2f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 11:58:41.113847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: warnings.warn(msg) Oct 04 11:58:41.113847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: Oct 04 11:58:41.116365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641fa917-ae81-4700-bd48-c7a370a2a2f0 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:58:41.239469 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 729/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 11:58:40 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 11:58:41.244685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e1c7d-39e5-40c9-99b4-3e822ecbc87f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:58:41.257417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e1c7d-39e5-40c9-99b4-3e822ecbc87f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:58:41.262408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6ed69-94db-4438-a17e-8a7e40c26e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:58:41.263037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6ed69-94db-4438-a17e-8a7e40c26e69 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:41.263320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6ed69-94db-4438-a17e-8a7e40c26e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:41.263594 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6ed69-94db-4438-a17e-8a7e40c26e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:41.264284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6ed69-94db-4438-a17e-8a7e40c26e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:58:41.265329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: WARNING py.warnings [None req-c9e6ed69-94db-4438-a17e-8a7e40c26e69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 11:58:41.265329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: warnings.warn(msg) Oct 04 11:58:41.265329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: Oct 04 11:58:41.266252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6ed69-94db-4438-a17e-8a7e40c26e69 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:58:41.283812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 770/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 11:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 11:58:41.288515 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fa917-ae81-4700-bd48-c7a370a2a2f0 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:58:41.300228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fa917-ae81-4700-bd48-c7a370a2a2f0 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:58:41.305102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b2ca3-a28e-4774-acaf-366858337394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:58:41.305725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b2ca3-a28e-4774-acaf-366858337394 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:41.306034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b2ca3-a28e-4774-acaf-366858337394 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:41.306308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b2ca3-a28e-4774-acaf-366858337394 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:41.306964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b2ca3-a28e-4774-acaf-366858337394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:58:41.308004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WARNING py.warnings [None req-618b2ca3-a28e-4774-acaf-366858337394 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 11:58:41.308004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: warnings.warn(msg) Oct 04 11:58:41.308004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: Oct 04 11:58:41.308945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b2ca3-a28e-4774-acaf-366858337394 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:58:41.324481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 730/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 11:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 11:58:41.329317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6ed69-94db-4438-a17e-8a7e40c26e69 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:58:41.340609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6ed69-94db-4438-a17e-8a7e40c26e69 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:58:41.347206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921e4b6-dac2-454f-9967-f091b103c7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:58:41.348275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921e4b6-dac2-454f-9967-f091b103c7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:41.348715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921e4b6-dac2-454f-9967-f091b103c7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:41.349037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921e4b6-dac2-454f-9967-f091b103c7c0 admin admin] PATH_INFO: `/v3/projects/cef801a0259d44febe57ba3e4be49d53/users/e154668e25044b769b86f13ac9175fc0/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:41.368096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921e4b6-dac2-454f-9967-f091b103c7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef801a0259d44febe57ba3e4be49d53, user_id=e154668e25044b769b86f13ac9175fc0, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:58:41.369384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: WARNING py.warnings [None req-d921e4b6-dac2-454f-9967-f091b103c7c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 11:58:41.369384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: warnings.warn(msg) Oct 04 11:58:41.369384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: Oct 04 11:58:41.371622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921e4b6-dac2-454f-9967-f091b103c7c0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:58:41.462334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 771/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 11:58:41 2022] PUT /identity/v3/projects/cef801a0259d44febe57ba3e4be49d53/users/e154668e25044b769b86f13ac9175fc0/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:58:41.476573 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461ad75-1ea3-4f2f-ade6-5d9cfbff098d None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:41.477210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461ad75-1ea3-4f2f-ade6-5d9cfbff098d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:41.477210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461ad75-1ea3-4f2f-ade6-5d9cfbff098d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:41.514111 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-c461ad75-1ea3-4f2f-ade6-5d9cfbff098d None None] MFA Rules not processed for user `e154668e25044b769b86f13ac9175fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:58:41.515528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c461ad75-1ea3-4f2f-ade6-5d9cfbff098d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:58:41.615339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 731/1502] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 11:58:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:58:41.630263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ff69a-7dd2-4555-a3dc-39b82eb791dc None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:41.630616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ff69a-7dd2-4555-a3dc-39b82eb791dc None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:41.630939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ff69a-7dd2-4555-a3dc-39b82eb791dc None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:41.632626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 772/1503] 10.222.0.15 () {54 vars in 963 bytes} [Tue Oct 4 11:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 11:58:41.639006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6937813-8a6b-4cfe-8544-80dec66468ff None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:41.639396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6937813-8a6b-4cfe-8544-80dec66468ff None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:41.639710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6937813-8a6b-4cfe-8544-80dec66468ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:41.737190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e6937813-8a6b-4cfe-8544-80dec66468ff None None] MFA Rules not processed for user `496a2e8bf591437888ea82161ca4d5ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:58:41.739584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e6937813-8a6b-4cfe-8544-80dec66468ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:58:41.854640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 732/1504] 10.222.0.15 () {58 vars in 1060 bytes} [Tue Oct 4 11:58:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 11:58:41.865693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ff69a-7dd2-4555-a3dc-39b82eb791dc None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:58:41.878242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ff69a-7dd2-4555-a3dc-39b82eb791dc None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:58:41.882373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03103b00-3703-4118-8b60-f0739bcbd468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:58:41.883197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03103b00-3703-4118-8b60-f0739bcbd468 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:41.883524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03103b00-3703-4118-8b60-f0739bcbd468 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:41.883841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03103b00-3703-4118-8b60-f0739bcbd468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:41.895505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03103b00-3703-4118-8b60-f0739bcbd468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:58:41.897104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03103b00-3703-4118-8b60-f0739bcbd468 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:58:41.908914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 773/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:58:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:58:41.967562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39ac3a-b8dd-43fc-8f71-4135c25598b9 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:41.967839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39ac3a-b8dd-43fc-8f71-4135c25598b9 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:41.968106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39ac3a-b8dd-43fc-8f71-4135c25598b9 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:41.969491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 733/1506] 10.222.0.15 () {52 vars in 830 bytes} [Tue Oct 4 11:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 11:58:41.974913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03103b00-3703-4118-8b60-f0739bcbd468 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:58:42.075422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03103b00-3703-4118-8b60-f0739bcbd468 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:58:42.080894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:58:42.081545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:42.081824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:42.082095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] PATH_INFO: `/v3/users/e154668e25044b769b86f13ac9175fc0` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:42.083341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e154668e25044b769b86f13ac9175fc0)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:58:42.084402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: WARNING py.warnings [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 11:58:42.084402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: warnings.warn(msg) Oct 04 11:58:42.084402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: Oct 04 11:58:42.085623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:58:42.441537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e154668e25044b769b86f13ac9175fc0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 11:58:42.442506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e154668e25044b769b86f13ac9175fc0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 11:58:42.461585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e154668e25044b769b86f13ac9175fc0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 11:58:42.462352 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e154668e25044b769b86f13ac9175fc0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 11:58:42.609487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e154668e25044b769b86f13ac9175fc0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 11:58:42.636612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:58:42.646119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 774/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 11:58:41 2022] DELETE /identity/v3/users/e154668e25044b769b86f13ac9175fc0 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:58:42.653191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749af6de-3dcc-405a-b11d-d70d21c8d03d None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:42.653861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749af6de-3dcc-405a-b11d-d70d21c8d03d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:42.653861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749af6de-3dcc-405a-b11d-d70d21c8d03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:42.677089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-749af6de-3dcc-405a-b11d-d70d21c8d03d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 11:58:42.678342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-749af6de-3dcc-405a-b11d-d70d21c8d03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:58:42.689229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:58:42.747289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 734/1508] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 11:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 11:58:42.758841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:58:42.760398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:58:42.851997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d5333-71fa-47d9-b4c3-d74ea2902a77 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:58:42.855977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d6278-1455-4375-ac8c-5db56f64ab9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:58:42.856623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d6278-1455-4375-ac8c-5db56f64ab9a service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:42.860859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d6278-1455-4375-ac8c-5db56f64ab9a service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:42.860859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d6278-1455-4375-ac8c-5db56f64ab9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:42.867828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d6278-1455-4375-ac8c-5db56f64ab9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:58:42.869347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d6278-1455-4375-ac8c-5db56f64ab9a service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:58:42.881330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 775/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 11:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 11:58:43.565290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749af6de-3dcc-405a-b11d-d70d21c8d03d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 11:58:43.569703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-749af6de-3dcc-405a-b11d-d70d21c8d03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 11:58:43.595809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749af6de-3dcc-405a-b11d-d70d21c8d03d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 11:58:43.606834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 11:58:43.608002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 11:58:43.609709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 11:58:43.610535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] PATH_INFO: `/v3/projects/cef801a0259d44febe57ba3e4be49d53` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 11:58:43.612360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cef801a0259d44febe57ba3e4be49d53)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 11:58:43.613548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WARNING py.warnings [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 11:58:43.613548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: warnings.warn(msg) Oct 04 11:58:43.613548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: Oct 04 11:58:43.614842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 11:58:43.827868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cef801a0259d44febe57ba3e4be49d53', 'request_id': 'req-d955d1dd-a68e-49d8-81a9-058ea9390a1e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 11:58:43.835516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cef801a0259d44febe57ba3e4be49d53', 'request_id': 'req-d955d1dd-a68e-49d8-81a9-058ea9390a1e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 11:58:43.851367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cef801a0259d44febe57ba3e4be49d53', 'request_id': 'req-d955d1dd-a68e-49d8-81a9-058ea9390a1e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 11:58:44.084936 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] The token cache is being invalidate because project cef801a0259d44febe57ba3e4be49d53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 11:58:44.085435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 11:58:44.085930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d955d1dd-a68e-49d8-81a9-058ea9390a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 11:58:44.087635 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 735/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 11:58:43 2022] DELETE /identity/v3/projects/cef801a0259d44febe57ba3e4be49d53 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 11:58:52.907623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 11:58:54.081044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:00:16.156249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056c570-a8a3-4671-b5c3-ec456f2415f6 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:16.157088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056c570-a8a3-4671-b5c3-ec456f2415f6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:16.157088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056c570-a8a3-4671-b5c3-ec456f2415f6 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:16.158406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 776/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:00:16.164927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50665f86-ba6b-4e43-8759-2dd9bedf71f2 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:16.165823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50665f86-ba6b-4e43-8759-2dd9bedf71f2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:16.165823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50665f86-ba6b-4e43-8759-2dd9bedf71f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:16.189380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-50665f86-ba6b-4e43-8759-2dd9bedf71f2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:00:16.196352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-50665f86-ba6b-4e43-8759-2dd9bedf71f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:16.260639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 736/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:00:16.418941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056c570-a8a3-4671-b5c3-ec456f2415f6 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:00:16.422665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e056c570-a8a3-4671-b5c3-ec456f2415f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:16.553662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056c570-a8a3-4671-b5c3-ec456f2415f6 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:00:16.560541 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638b15f-935b-4cf7-93c1-ab9e698d8c1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:00:16.561185 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638b15f-935b-4cf7-93c1-ab9e698d8c1f service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:16.561498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638b15f-935b-4cf7-93c1-ab9e698d8c1f service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:16.561734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638b15f-935b-4cf7-93c1-ab9e698d8c1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:16.571426 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638b15f-935b-4cf7-93c1-ab9e698d8c1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:00:16.573252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638b15f-935b-4cf7-93c1-ab9e698d8c1f service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:00:16.584067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 777/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:00:17.813678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61524c9d-2348-4fda-a916-8de093c36e60 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:17.814355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61524c9d-2348-4fda-a916-8de093c36e60 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:17.814355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61524c9d-2348-4fda-a916-8de093c36e60 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:17.815666 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 737/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:00:17.823026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4306f77-5b9a-4eb8-ba17-d9f10946f4d5 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:17.824001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4306f77-5b9a-4eb8-ba17-d9f10946f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:17.824001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4306f77-5b9a-4eb8-ba17-d9f10946f4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:17.845859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e4306f77-5b9a-4eb8-ba17-d9f10946f4d5 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:00:17.847099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e4306f77-5b9a-4eb8-ba17-d9f10946f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:17.855557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 778/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:00:18.147977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61524c9d-2348-4fda-a916-8de093c36e60 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:00:18.163887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61524c9d-2348-4fda-a916-8de093c36e60 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:00:18.170896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd01e0-8e52-4156-b0aa-2b01134ef7f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:00:18.171684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd01e0-8e52-4156-b0aa-2b01134ef7f3 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:18.172089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd01e0-8e52-4156-b0aa-2b01134ef7f3 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:18.172504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd01e0-8e52-4156-b0aa-2b01134ef7f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:18.182693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd01e0-8e52-4156-b0aa-2b01134ef7f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:00:18.184588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd01e0-8e52-4156-b0aa-2b01134ef7f3 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:00:18.195791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 738/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:00:21.373880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4335ce0-6412-4f2f-a218-c2d6e995ac98 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:21.374595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4335ce0-6412-4f2f-a218-c2d6e995ac98 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:21.374595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4335ce0-6412-4f2f-a218-c2d6e995ac98 None None] PATH_INFO: `/v3` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:21.376061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 779/1517] 10.222.0.43 () {54 vars in 925 bytes} [Tue Oct 4 12:00:21 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:00:21.387522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9669141-5e2a-4204-b099-6c6ed645f6ed None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:21.388817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9669141-5e2a-4204-b099-6c6ed645f6ed None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:21.388817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9669141-5e2a-4204-b099-6c6ed645f6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:21.449196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e9669141-5e2a-4204-b099-6c6ed645f6ed None None] MFA Rules not processed for user `16374dbb6d084ddfa3b49afd19c7c325`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:00:21.451215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e9669141-5e2a-4204-b099-6c6ed645f6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:21.565716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 739/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Tue Oct 4 12:00:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:00:26.214050 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333d9e0-1b2f-4bcf-a67d-bb8c1a15c0af None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:26.214705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333d9e0-1b2f-4bcf-a67d-bb8c1a15c0af None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:26.214705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333d9e0-1b2f-4bcf-a67d-bb8c1a15c0af None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:26.216717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 780/1519] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 4 12:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:00:26.224900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a5fc3-090e-475c-bb30-327c5ae755eb None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:26.225802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a5fc3-090e-475c-bb30-327c5ae755eb None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:26.225802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a5fc3-090e-475c-bb30-327c5ae755eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:26.249418 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-be6a5fc3-090e-475c-bb30-327c5ae755eb None None] MFA Rules not processed for user `16374dbb6d084ddfa3b49afd19c7c325`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:00:26.250668 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-be6a5fc3-090e-475c-bb30-327c5ae755eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:26.260037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 740/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 4 12:00:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:00:26.269014 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333d9e0-1b2f-4bcf-a67d-bb8c1a15c0af None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:00:26.284441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333d9e0-1b2f-4bcf-a67d-bb8c1a15c0af None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:00:26.290277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdadce1-24cb-44cd-a92e-384adeb3f992 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:00:26.290892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdadce1-24cb-44cd-a92e-384adeb3f992 service placement] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:26.291174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdadce1-24cb-44cd-a92e-384adeb3f992 service placement] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:26.291447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdadce1-24cb-44cd-a92e-384adeb3f992 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:26.302415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdadce1-24cb-44cd-a92e-384adeb3f992 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:00:26.306048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdadce1-24cb-44cd-a92e-384adeb3f992 service placement] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:00:26.326502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 781/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:00:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:00:29.768788 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32702b31-8ce8-4de4-9985-0860d090013e None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:29.769529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32702b31-8ce8-4de4-9985-0860d090013e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:29.769529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32702b31-8ce8-4de4-9985-0860d090013e None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:29.770679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 741/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:00:29.778508 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8f376-69cc-4a36-a2c7-7dbd9fe6dbe0 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:29.779091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8f376-69cc-4a36-a2c7-7dbd9fe6dbe0 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:29.779091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8f376-69cc-4a36-a2c7-7dbd9fe6dbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:29.800438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-04a8f376-69cc-4a36-a2c7-7dbd9fe6dbe0 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:00:29.801558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-04a8f376-69cc-4a36-a2c7-7dbd9fe6dbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:29.809595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 782/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:00:29.971283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32702b31-8ce8-4de4-9985-0860d090013e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:00:29.988006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32702b31-8ce8-4de4-9985-0860d090013e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:00:29.994184 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbea02-fc46-4e84-af50-c415d6d78908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:00:29.994886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbea02-fc46-4e84-af50-c415d6d78908 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:29.995279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbea02-fc46-4e84-af50-c415d6d78908 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:29.995583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbea02-fc46-4e84-af50-c415d6d78908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:30.005816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbea02-fc46-4e84-af50-c415d6d78908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:00:30.007555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbea02-fc46-4e84-af50-c415d6d78908 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:00:30.018365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 742/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:00:31.152779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba919469-b308-482e-8865-d1d2f58c7934 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:31.153390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba919469-b308-482e-8865-d1d2f58c7934 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:31.153390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba919469-b308-482e-8865-d1d2f58c7934 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:31.154726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 783/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:00:31.162936 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0093d-2e11-45e7-bdbf-5ace182d2aed None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:31.163893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0093d-2e11-45e7-bdbf-5ace182d2aed None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:31.163893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0093d-2e11-45e7-bdbf-5ace182d2aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:31.186012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-aef0093d-2e11-45e7-bdbf-5ace182d2aed None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:00:31.187264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-aef0093d-2e11-45e7-bdbf-5ace182d2aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:31.195320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 743/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:00:31.351770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba919469-b308-482e-8865-d1d2f58c7934 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:00:31.377902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba919469-b308-482e-8865-d1d2f58c7934 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:00:31.388580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0333598-128a-4874-a420-2b96ad5feede service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:00:31.389697 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0333598-128a-4874-a420-2b96ad5feede service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:31.390194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0333598-128a-4874-a420-2b96ad5feede service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:31.390700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0333598-128a-4874-a420-2b96ad5feede service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:31.404839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0333598-128a-4874-a420-2b96ad5feede service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:00:31.406541 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0333598-128a-4874-a420-2b96ad5feede service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:00:31.417950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 784/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:00:35.577232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f90480-572e-4cf6-85d0-c8f907d0210d None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:35.578000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f90480-572e-4cf6-85d0-c8f907d0210d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:35.578287 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f90480-572e-4cf6-85d0-c8f907d0210d None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:35.580909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 744/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:00:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:00:35.589966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024a461-5444-493c-a509-5262b25df71f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:35.591144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024a461-5444-493c-a509-5262b25df71f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:35.591144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024a461-5444-493c-a509-5262b25df71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:35.613194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b024a461-5444-493c-a509-5262b25df71f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:00:35.614440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b024a461-5444-493c-a509-5262b25df71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:35.622657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 785/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:00:35.922883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f90480-572e-4cf6-85d0-c8f907d0210d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:00:35.924777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-61f90480-572e-4cf6-85d0-c8f907d0210d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:35.958376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:00:35.993317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f90480-572e-4cf6-85d0-c8f907d0210d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:00:35.999692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b46fd-66d0-4c03-8046-5cfa667b6b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:00:36.000797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b46fd-66d0-4c03-8046-5cfa667b6b01 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:36.001269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b46fd-66d0-4c03-8046-5cfa667b6b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:36.001653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b46fd-66d0-4c03-8046-5cfa667b6b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:36.016966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b46fd-66d0-4c03-8046-5cfa667b6b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:00:36.019791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b46fd-66d0-4c03-8046-5cfa667b6b01 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:00:36.038371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 745/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:00:38.401574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3e615-ef76-46ac-a36c-dcadb1a490ce None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:38.402313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3e615-ef76-46ac-a36c-dcadb1a490ce None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:38.402313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3e615-ef76-46ac-a36c-dcadb1a490ce None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:38.404124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 786/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:00:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:00:38.410812 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed5499-5f38-4247-993c-dfae81e4b524 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:38.411674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed5499-5f38-4247-993c-dfae81e4b524 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:38.411674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed5499-5f38-4247-993c-dfae81e4b524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:38.433620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-86ed5499-5f38-4247-993c-dfae81e4b524 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:00:38.435045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-86ed5499-5f38-4247-993c-dfae81e4b524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:38.444267 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 746/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:00:38.750022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3e615-ef76-46ac-a36c-dcadb1a490ce None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:00:38.762988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3e615-ef76-46ac-a36c-dcadb1a490ce None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:00:38.767099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25003a25-87ff-4caa-8554-178308f257ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:00:38.767724 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25003a25-87ff-4caa-8554-178308f257ae service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:38.768010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25003a25-87ff-4caa-8554-178308f257ae service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:38.768327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25003a25-87ff-4caa-8554-178308f257ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:38.779237 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25003a25-87ff-4caa-8554-178308f257ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:00:38.781024 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25003a25-87ff-4caa-8554-178308f257ae service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:00:38.795018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 787/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:00:42.642093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db043e4d-a23e-464d-a34a-8ea8692385a8 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:42.643016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db043e4d-a23e-464d-a34a-8ea8692385a8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:42.643016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db043e4d-a23e-464d-a34a-8ea8692385a8 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:42.644023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 747/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:00:42.651238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749d72c-a3b0-4639-8c51-6cc65ae07632 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:42.651884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749d72c-a3b0-4639-8c51-6cc65ae07632 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:42.652251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749d72c-a3b0-4639-8c51-6cc65ae07632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:42.687113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-3749d72c-a3b0-4639-8c51-6cc65ae07632 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:00:42.689162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3749d72c-a3b0-4639-8c51-6cc65ae07632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:42.703576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 788/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:00:43.099492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db043e4d-a23e-464d-a34a-8ea8692385a8 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:00:43.101907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-db043e4d-a23e-464d-a34a-8ea8692385a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:43.172005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db043e4d-a23e-464d-a34a-8ea8692385a8 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:00:43.176134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf55a19-44b3-4e9d-8a56-3d99c7b640c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:00:43.176790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf55a19-44b3-4e9d-8a56-3d99c7b640c0 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:43.177070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf55a19-44b3-4e9d-8a56-3d99c7b640c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:43.177345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf55a19-44b3-4e9d-8a56-3d99c7b640c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:43.188542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf55a19-44b3-4e9d-8a56-3d99c7b640c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:00:43.190247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf55a19-44b3-4e9d-8a56-3d99c7b640c0 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:00:43.205678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 748/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:00:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:00:45.614154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7c6ec-8eff-47a3-a264-dfef9fc26e9c None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:45.614844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7c6ec-8eff-47a3-a264-dfef9fc26e9c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:45.614979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7c6ec-8eff-47a3-a264-dfef9fc26e9c None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:45.616469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 789/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:00:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:00:45.623564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e96738-24ab-4ceb-870c-aa7b2e2add60 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:45.624326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e96738-24ab-4ceb-870c-aa7b2e2add60 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:45.624326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e96738-24ab-4ceb-870c-aa7b2e2add60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:45.648585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b4e96738-24ab-4ceb-870c-aa7b2e2add60 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:00:45.650121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b4e96738-24ab-4ceb-870c-aa7b2e2add60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:45.659784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 749/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:00:45.988492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7c6ec-8eff-47a3-a264-dfef9fc26e9c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:00:45.990457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-51d7c6ec-8eff-47a3-a264-dfef9fc26e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:00:45.999751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=77 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:00:46.006392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7c6ec-8eff-47a3-a264-dfef9fc26e9c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:00:46.010279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8e421-17aa-46ab-b13a-28b3a781acb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:00:46.010930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e421-17aa-46ab-b13a-28b3a781acb3 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:00:46.011207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e421-17aa-46ab-b13a-28b3a781acb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:00:46.011483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e421-17aa-46ab-b13a-28b3a781acb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:00:46.022317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8e421-17aa-46ab-b13a-28b3a781acb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:00:46.024327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8e421-17aa-46ab-b13a-28b3a781acb3 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:00:46.038695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 790/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:00:55.641918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=47 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:00:56.053586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:21.545354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3614a-4e98-480d-8c96-bd71cf0e49a2 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:21.546431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3614a-4e98-480d-8c96-bd71cf0e49a2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:21.546928 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3614a-4e98-480d-8c96-bd71cf0e49a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:21.574838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b8e3614a-4e98-480d-8c96-bd71cf0e49a2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:21.576557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b8e3614a-4e98-480d-8c96-bd71cf0e49a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:21.588765 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 750/1540] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:02:21.597280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f26b2f-1229-4f79-9606-3fa4997d2305 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:21.597909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f26b2f-1229-4f79-9606-3fa4997d2305 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:21.597909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f26b2f-1229-4f79-9606-3fa4997d2305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:21.631525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-69f26b2f-1229-4f79-9606-3fa4997d2305 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:21.633364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-69f26b2f-1229-4f79-9606-3fa4997d2305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:21.657868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 791/1541] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:21.663607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3614a-4e98-480d-8c96-bd71cf0e49a2 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:21.679179 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3614a-4e98-480d-8c96-bd71cf0e49a2 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:21.696488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334fdb1-5875-4e19-8879-b292d6549005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:21.697556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334fdb1-5875-4e19-8879-b292d6549005 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:21.698073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334fdb1-5875-4e19-8879-b292d6549005 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:21.698620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334fdb1-5875-4e19-8879-b292d6549005 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:21.699515 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334fdb1-5875-4e19-8879-b292d6549005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:21.700077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334fdb1-5875-4e19-8879-b292d6549005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:21.702136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334fdb1-5875-4e19-8879-b292d6549005 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:21.726694 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 751/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:21.734794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f26b2f-1229-4f79-9606-3fa4997d2305 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:21.755500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f26b2f-1229-4f79-9606-3fa4997d2305 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:21.765864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed63af-7d62-4eb0-b543-62b4f54276f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:21.766499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed63af-7d62-4eb0-b543-62b4f54276f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:21.766781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed63af-7d62-4eb0-b543-62b4f54276f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:21.767056 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed63af-7d62-4eb0-b543-62b4f54276f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:21.767719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed63af-7d62-4eb0-b543-62b4f54276f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:21.769765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed63af-7d62-4eb0-b543-62b4f54276f5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:21.844435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8e471-76b4-4cd2-8b65-a75fd68c3584 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:21.844745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8e471-76b4-4cd2-8b65-a75fd68c3584 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:21.845079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8e471-76b4-4cd2-8b65-a75fd68c3584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:21.889743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f6a8e471-76b4-4cd2-8b65-a75fd68c3584 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:21.891107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f6a8e471-76b4-4cd2-8b65-a75fd68c3584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:21.906293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 752/1543] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:02:21.923733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70470d0b-0efb-4216-8c84-2450a8f66908 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:21.929621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70470d0b-0efb-4216-8c84-2450a8f66908 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:21.929621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70470d0b-0efb-4216-8c84-2450a8f66908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:21.930006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 792/1544] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:21 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:02:21.932579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed63af-7d62-4eb0-b543-62b4f54276f5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:21.961300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-70470d0b-0efb-4216-8c84-2450a8f66908 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:21.962618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-70470d0b-0efb-4216-8c84-2450a8f66908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.077190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed63af-7d62-4eb0-b543-62b4f54276f5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.094420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647cc35-eeb7-41f0-a2ba-67d864b5a63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.095059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647cc35-eeb7-41f0-a2ba-67d864b5a63b admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.095394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647cc35-eeb7-41f0-a2ba-67d864b5a63b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.097837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 753/1545] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:02:22.104892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647cc35-eeb7-41f0-a2ba-67d864b5a63b admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.105701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647cc35-eeb7-41f0-a2ba-67d864b5a63b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.107214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: WARNING py.warnings [None req-8647cc35-eeb7-41f0-a2ba-67d864b5a63b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:02:22.107214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: warnings.warn(msg) Oct 04 12:02:22.107214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: Oct 04 12:02:22.108430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95796a0d-5d8b-4d2f-b16e-23cac18758fa None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.108775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95796a0d-5d8b-4d2f-b16e-23cac18758fa None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.109043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95796a0d-5d8b-4d2f-b16e-23cac18758fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.112666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647cc35-eeb7-41f0-a2ba-67d864b5a63b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:22.165270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-95796a0d-5d8b-4d2f-b16e-23cac18758fa None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.166565 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-95796a0d-5d8b-4d2f-b16e-23cac18758fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.185119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 754/1546] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.186011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95796a0d-5d8b-4d2f-b16e-23cac18758fa None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.198262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95796a0d-5d8b-4d2f-b16e-23cac18758fa None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.206878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc509979-163a-43c5-a4e2-8b99990d1ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.207518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc509979-163a-43c5-a4e2-8b99990d1ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.207790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc509979-163a-43c5-a4e2-8b99990d1ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.208068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc509979-163a-43c5-a4e2-8b99990d1ee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.208788 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc509979-163a-43c5-a4e2-8b99990d1ee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:22.209087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc509979-163a-43c5-a4e2-8b99990d1ee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.211048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc509979-163a-43c5-a4e2-8b99990d1ee9 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:22.230586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 755/1547] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:22.232126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291f80d-5ba5-4f25-a6b5-82c054f562a0 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.232447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291f80d-5ba5-4f25-a6b5-82c054f562a0 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.232735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291f80d-5ba5-4f25-a6b5-82c054f562a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.256236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 793/1548] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:21 2022] POST /identity/v3/users => generated 293 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:22.261677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225b3bd-5416-4d44-b275-85768208a64b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.261677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225b3bd-5416-4d44-b275-85768208a64b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.261677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225b3bd-5416-4d44-b275-85768208a64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.264996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-2291f80d-5ba5-4f25-a6b5-82c054f562a0 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.269781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2291f80d-5ba5-4f25-a6b5-82c054f562a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.284193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 756/1549] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.284727 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291f80d-5ba5-4f25-a6b5-82c054f562a0 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.302303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291f80d-5ba5-4f25-a6b5-82c054f562a0 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.307588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e74d6-1c00-49b1-9022-bf7504ad6cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.308261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e74d6-1c00-49b1-9022-bf7504ad6cec admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.308464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f225b3bd-5416-4d44-b275-85768208a64b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.308644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e74d6-1c00-49b1-9022-bf7504ad6cec admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.308800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e74d6-1c00-49b1-9022-bf7504ad6cec admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.309454 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e74d6-1c00-49b1-9022-bf7504ad6cec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.310520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WARNING py.warnings [None req-582e74d6-1c00-49b1-9022-bf7504ad6cec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:02:22.310520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: warnings.warn(msg) Oct 04 12:02:22.310520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: Oct 04 12:02:22.310979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f225b3bd-5416-4d44-b275-85768208a64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.311751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e74d6-1c00-49b1-9022-bf7504ad6cec admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:22.327860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 794/1550] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.330281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a8e5-d954-4ab2-b674-8d9e1e42493f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.330839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a8e5-d954-4ab2-b674-8d9e1e42493f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.331325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a8e5-d954-4ab2-b674-8d9e1e42493f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.368983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-36f2a8e5-d954-4ab2-b674-8d9e1e42493f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.371281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-36f2a8e5-d954-4ab2-b674-8d9e1e42493f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.387443 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 795/1551] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.389111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a8e5-d954-4ab2-b674-8d9e1e42493f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.407654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a8e5-d954-4ab2-b674-8d9e1e42493f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.416073 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ae5ab-81ad-443f-8858-735abe1819c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.418797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ae5ab-81ad-443f-8858-735abe1819c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.418797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ae5ab-81ad-443f-8858-735abe1819c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.418797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ae5ab-81ad-443f-8858-735abe1819c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.419439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ae5ab-81ad-443f-8858-735abe1819c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.422640 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ae5ab-81ad-443f-8858-735abe1819c4 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:22.435555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 757/1552] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:22.435555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda1c1-9cc2-4b3b-985f-874be4070433 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.435555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda1c1-9cc2-4b3b-985f-874be4070433 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.435555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda1c1-9cc2-4b3b-985f-874be4070433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.447704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 796/1553] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:22.450695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9435f9-609e-4a3d-8a2c-4ce76ea1f85e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.451346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9435f9-609e-4a3d-8a2c-4ce76ea1f85e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.451695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9435f9-609e-4a3d-8a2c-4ce76ea1f85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.459112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-45eda1c1-9cc2-4b3b-985f-874be4070433 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.461010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-45eda1c1-9cc2-4b3b-985f-874be4070433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.483814 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-bc9435f9-609e-4a3d-8a2c-4ce76ea1f85e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.486033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-bc9435f9-609e-4a3d-8a2c-4ce76ea1f85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.531541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 758/1554] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.532431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda1c1-9cc2-4b3b-985f-874be4070433 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.544247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda1c1-9cc2-4b3b-985f-874be4070433 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.549309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3e75f-727c-421e-b684-4689e1a55775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.550425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3e75f-727c-421e-b684-4689e1a55775 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.550425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3e75f-727c-421e-b684-4689e1a55775 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.550789 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3e75f-727c-421e-b684-4689e1a55775 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.551151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3e75f-727c-421e-b684-4689e1a55775 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:22.551461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3e75f-727c-421e-b684-4689e1a55775 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.553182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3e75f-727c-421e-b684-4689e1a55775 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:22.575544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 759/1555] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:22.575544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06245236-1ab2-4d8f-90f9-7e7ba43765ec None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.575544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06245236-1ab2-4d8f-90f9-7e7ba43765ec None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.575544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06245236-1ab2-4d8f-90f9-7e7ba43765ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.599465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-06245236-1ab2-4d8f-90f9-7e7ba43765ec None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.600964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-06245236-1ab2-4d8f-90f9-7e7ba43765ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.607006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 797/1556] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.609897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab743f-b952-4ad9-9b93-0108d436062f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.610376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab743f-b952-4ad9-9b93-0108d436062f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.610891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab743f-b952-4ad9-9b93-0108d436062f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.617929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 760/1557] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.620005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06245236-1ab2-4d8f-90f9-7e7ba43765ec None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.636770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06245236-1ab2-4d8f-90f9-7e7ba43765ec None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.642528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa50939-f838-417a-8d16-edc8da78c102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.643322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa50939-f838-417a-8d16-edc8da78c102 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.643541 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-ffab743f-b952-4ad9-9b93-0108d436062f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.643790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa50939-f838-417a-8d16-edc8da78c102 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.644216 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa50939-f838-417a-8d16-edc8da78c102 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.645152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa50939-f838-417a-8d16-edc8da78c102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.645364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ffab743f-b952-4ad9-9b93-0108d436062f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.648079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa50939-f838-417a-8d16-edc8da78c102 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:22.656189 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 798/1558] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.657803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:22.658759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97376b-3061-43f3-833d-875c4be41671 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.659240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97376b-3061-43f3-833d-875c4be41671 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.659712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97376b-3061-43f3-833d-875c4be41671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.686146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-6b97376b-3061-43f3-833d-875c4be41671 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.687724 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6b97376b-3061-43f3-833d-875c4be41671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.696551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 799/1559] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.697301 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97376b-3061-43f3-833d-875c4be41671 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.707975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97376b-3061-43f3-833d-875c4be41671 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.713818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94f65e-b07b-488c-a15f-15d5b5a4a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.714454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94f65e-b07b-488c-a15f-15d5b5a4a4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.714732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94f65e-b07b-488c-a15f-15d5b5a4a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.715006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94f65e-b07b-488c-a15f-15d5b5a4a4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.715675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94f65e-b07b-488c-a15f-15d5b5a4a4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.717372 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94f65e-b07b-488c-a15f-15d5b5a4a4d6 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:22.729508 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 800/1560] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:22.730955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2019b-b4e6-4f72-b0ef-f8b2e8bba0a0 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.731357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2019b-b4e6-4f72-b0ef-f8b2e8bba0a0 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.731576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2019b-b4e6-4f72-b0ef-f8b2e8bba0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.764913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 761/1561] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:22.765234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-bda2019b-b4e6-4f72-b0ef-f8b2e8bba0a0 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.765234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-bda2019b-b4e6-4f72-b0ef-f8b2e8bba0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.765545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa50939-f838-417a-8d16-edc8da78c102 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.775159 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 801/1562] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.777060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2019b-b4e6-4f72-b0ef-f8b2e8bba0a0 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.781219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa50939-f838-417a-8d16-edc8da78c102 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.792491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2019b-b4e6-4f72-b0ef-f8b2e8bba0a0 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.795756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89e712-673a-4f13-bb18-b9acd6637362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.797094 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89e712-673a-4f13-bb18-b9acd6637362 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.797617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89e712-673a-4f13-bb18-b9acd6637362 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.798026 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89e712-673a-4f13-bb18-b9acd6637362 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.799138 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89e712-673a-4f13-bb18-b9acd6637362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.800649 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7bc0b-3c6c-4395-9a8a-22baeeab89b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.801612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7bc0b-3c6c-4395-9a8a-22baeeab89b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.802070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7bc0b-3c6c-4395-9a8a-22baeeab89b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.802542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7bc0b-3c6c-4395-9a8a-22baeeab89b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.802740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89e712-673a-4f13-bb18-b9acd6637362 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:22.803617 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7bc0b-3c6c-4395-9a8a-22baeeab89b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:22.804142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7bc0b-3c6c-4395-9a8a-22baeeab89b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.805991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: WARNING py.warnings [None req-71c7bc0b-3c6c-4395-9a8a-22baeeab89b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:02:22.805991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: warnings.warn(msg) Oct 04 12:02:22.805991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: Oct 04 12:02:22.811984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7bc0b-3c6c-4395-9a8a-22baeeab89b9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:22.811984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:22.838346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 802/1563] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:22.838346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c27eb1-7785-4d04-8321-99ce1a79d63e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.838346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c27eb1-7785-4d04-8321-99ce1a79d63e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.838346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c27eb1-7785-4d04-8321-99ce1a79d63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.852934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b8c27eb1-7785-4d04-8321-99ce1a79d63e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.854070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b8c27eb1-7785-4d04-8321-99ce1a79d63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.862766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 803/1564] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.863645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c27eb1-7785-4d04-8321-99ce1a79d63e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.874356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c27eb1-7785-4d04-8321-99ce1a79d63e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.881193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd1bf4-e569-460f-ab9a-697985c07754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.882471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd1bf4-e569-460f-ab9a-697985c07754 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.882793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd1bf4-e569-460f-ab9a-697985c07754 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.883111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd1bf4-e569-460f-ab9a-697985c07754 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.883830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd1bf4-e569-460f-ab9a-697985c07754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:22.884201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd1bf4-e569-460f-ab9a-697985c07754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.886320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd1bf4-e569-460f-ab9a-697985c07754 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:22.900529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 804/1565] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:22.913525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aa625-eddc-4e2d-a007-a0d0e1db9bdb None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.913525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aa625-eddc-4e2d-a007-a0d0e1db9bdb None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.913525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aa625-eddc-4e2d-a007-a0d0e1db9bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.925140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f07aa625-eddc-4e2d-a007-a0d0e1db9bdb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:22.926403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f07aa625-eddc-4e2d-a007-a0d0e1db9bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:22.939775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 805/1566] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:22.939775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aa625-eddc-4e2d-a007-a0d0e1db9bdb None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.947276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07aa625-eddc-4e2d-a007-a0d0e1db9bdb None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:22.955933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 762/1567] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:22 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:22.956266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74cdb-8c80-481d-aaca-a3ef284e3b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:22.956266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74cdb-8c80-481d-aaca-a3ef284e3b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:22.956266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74cdb-8c80-481d-aaca-a3ef284e3b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:22.956266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74cdb-8c80-481d-aaca-a3ef284e3b6f admin admin] PATH_INFO: `/v3/projects/bd2c6c3369b14759bae51f7cff4549b0/users/0f7fc9f080e94b7f904e020be896b5a3/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:22.956880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89e712-673a-4f13-bb18-b9acd6637362 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:22.979326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a74cdb-8c80-481d-aaca-a3ef284e3b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd2c6c3369b14759bae51f7cff4549b0, user_id=0f7fc9f080e94b7f904e020be896b5a3, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:22.985004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a74cdb-8c80-481d-aaca-a3ef284e3b6f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.040400 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89e712-673a-4f13-bb18-b9acd6637362 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.045555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e833a6-cc0a-42b7-8057-38793389eed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.046455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e833a6-cc0a-42b7-8057-38793389eed6 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.046587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e833a6-cc0a-42b7-8057-38793389eed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.046778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e833a6-cc0a-42b7-8057-38793389eed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.047443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e833a6-cc0a-42b7-8057-38793389eed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.049191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e833a6-cc0a-42b7-8057-38793389eed6 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.062286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 763/1568] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:23.063173 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e833a6-cc0a-42b7-8057-38793389eed6 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:23.074536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e833a6-cc0a-42b7-8057-38793389eed6 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.079957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4abab-6064-4b71-a7c7-c1a674752beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.080640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4abab-6064-4b71-a7c7-c1a674752beb admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.080921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4abab-6064-4b71-a7c7-c1a674752beb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.081196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4abab-6064-4b71-a7c7-c1a674752beb admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.081866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4abab-6064-4b71-a7c7-c1a674752beb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:23.082171 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4abab-6064-4b71-a7c7-c1a674752beb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.083932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4abab-6064-4b71-a7c7-c1a674752beb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.099055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 764/1569] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:23.099800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4abab-6064-4b71-a7c7-c1a674752beb admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:23.110276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4abab-6064-4b71-a7c7-c1a674752beb admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.115644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cdd6-5d60-4815-a2e5-107cbc2c36ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.121575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cdd6-5d60-4815-a2e5-107cbc2c36ca admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.121575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cdd6-5d60-4815-a2e5-107cbc2c36ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.121575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463cdd6-5d60-4815-a2e5-107cbc2c36ca admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.121575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463cdd6-5d60-4815-a2e5-107cbc2c36ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.121575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463cdd6-5d60-4815-a2e5-107cbc2c36ca admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.161140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 806/1570] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:22 2022] PUT /identity/v3/projects/bd2c6c3369b14759bae51f7cff4549b0/users/0f7fc9f080e94b7f904e020be896b5a3/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:23.163474 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74cdb-8c80-481d-aaca-a3ef284e3b6f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:23.242583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74cdb-8c80-481d-aaca-a3ef284e3b6f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.247704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ee9f1-9276-4640-a0b2-a440538118b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.248372 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee9f1-9276-4640-a0b2-a440538118b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.249363 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee9f1-9276-4640-a0b2-a440538118b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.249363 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee9f1-9276-4640-a0b2-a440538118b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.249633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ee9f1-9276-4640-a0b2-a440538118b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:23.249847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ee9f1-9276-4640-a0b2-a440538118b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.251491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ee9f1-9276-4640-a0b2-a440538118b3 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.269800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 807/1571] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:23.271324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ee9f1-9276-4640-a0b2-a440538118b3 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:23.286430 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ee9f1-9276-4640-a0b2-a440538118b3 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.294419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bacc2-d28e-4ac7-bc38-4731bba81df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.295451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bacc2-d28e-4ac7-bc38-4731bba81df5 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.295937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bacc2-d28e-4ac7-bc38-4731bba81df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.296461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bacc2-d28e-4ac7-bc38-4731bba81df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.297644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bacc2-d28e-4ac7-bc38-4731bba81df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.300631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bacc2-d28e-4ac7-bc38-4731bba81df5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.306832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 765/1572] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:23 2022] POST /identity/v3/projects => generated 403 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:23.306832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cdd6-5d60-4815-a2e5-107cbc2c36ca admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:23.390148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463cdd6-5d60-4815-a2e5-107cbc2c36ca admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.395656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c18de-7430-4890-b62d-1101996688bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.396478 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c18de-7430-4890-b62d-1101996688bb admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.396824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c18de-7430-4890-b62d-1101996688bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.397345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c18de-7430-4890-b62d-1101996688bb admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.398536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c18de-7430-4890-b62d-1101996688bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:23.398955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c18de-7430-4890-b62d-1101996688bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.400999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c18de-7430-4890-b62d-1101996688bb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.422415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 766/1573] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:23.423623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c18de-7430-4890-b62d-1101996688bb admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:23.437211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c18de-7430-4890-b62d-1101996688bb admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.443121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33304fa-4c6a-4a18-9eb7-a84ea0fefd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.448357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33304fa-4c6a-4a18-9eb7-a84ea0fefd4c admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.448357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33304fa-4c6a-4a18-9eb7-a84ea0fefd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.448357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33304fa-4c6a-4a18-9eb7-a84ea0fefd4c admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.448357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33304fa-4c6a-4a18-9eb7-a84ea0fefd4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.448357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33304fa-4c6a-4a18-9eb7-a84ea0fefd4c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.449162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 808/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:23 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:23.450906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bacc2-d28e-4ac7-bc38-4731bba81df5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:23.541787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 767/1575] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:23 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:23.541787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33304fa-4c6a-4a18-9eb7-a84ea0fefd4c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:23.554832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:23.562388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33304fa-4c6a-4a18-9eb7-a84ea0fefd4c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.565655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bacc2-d28e-4ac7-bc38-4731bba81df5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.571351 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d6bd6-fc51-48e8-b00c-c5b52e5e6968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.571957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e391f-9ca5-4c16-a88b-7eb8af8a28f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.572421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d6bd6-fc51-48e8-b00c-c5b52e5e6968 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.572861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e391f-9ca5-4c16-a88b-7eb8af8a28f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.572990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d6bd6-fc51-48e8-b00c-c5b52e5e6968 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.573257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e391f-9ca5-4c16-a88b-7eb8af8a28f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.573591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e391f-9ca5-4c16-a88b-7eb8af8a28f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.573721 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d6bd6-fc51-48e8-b00c-c5b52e5e6968 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.574360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e391f-9ca5-4c16-a88b-7eb8af8a28f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.574878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d6bd6-fc51-48e8-b00c-c5b52e5e6968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.576477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e391f-9ca5-4c16-a88b-7eb8af8a28f1 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.578840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d6bd6-fc51-48e8-b00c-c5b52e5e6968 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.588917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 809/1576] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:23.590104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f5b0-c812-4a23-b933-b511a1f671a7 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.590395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f5b0-c812-4a23-b933-b511a1f671a7 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.591416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f5b0-c812-4a23-b933-b511a1f671a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.613116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:23.625882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-efa1f5b0-c812-4a23-b933-b511a1f671a7 None None] MFA Rules not processed for user `0f7fc9f080e94b7f904e020be896b5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:23.628092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-efa1f5b0-c812-4a23-b933-b511a1f671a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:23.733348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 810/1577] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:23.734473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1f5b0-c812-4a23-b933-b511a1f671a7 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:23.751828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 768/1578] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:23 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:23.752149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1f5b0-c812-4a23-b933-b511a1f671a7 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.752364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d6bd6-fc51-48e8-b00c-c5b52e5e6968 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:23.841461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81d55-8377-496d-b890-9a90abe37805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.842585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81d55-8377-496d-b890-9a90abe37805 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.843208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81d55-8377-496d-b890-9a90abe37805 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.843653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81d55-8377-496d-b890-9a90abe37805 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.845391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e81d55-8377-496d-b890-9a90abe37805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.848105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e81d55-8377-496d-b890-9a90abe37805 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:23.853489 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d6bd6-fc51-48e8-b00c-c5b52e5e6968 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:23.860327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad776d-c145-43d9-8c92-9903ad6536a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:23.860581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad776d-c145-43d9-8c92-9903ad6536a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:23.861187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad776d-c145-43d9-8c92-9903ad6536a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:23.861512 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad776d-c145-43d9-8c92-9903ad6536a3 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:23.862751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad776d-c145-43d9-8c92-9903ad6536a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:23.868537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad776d-c145-43d9-8c92-9903ad6536a3 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:24.008426 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 811/1579] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:23 2022] POST /identity/v3/projects => generated 387 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:24.009774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81d55-8377-496d-b890-9a90abe37805 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.043020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 769/1580] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:23 2022] POST /identity/v3/users => generated 302 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:24.044100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad776d-c145-43d9-8c92-9903ad6536a3 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.056083 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad776d-c145-43d9-8c92-9903ad6536a3 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:24.118011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81d55-8377-496d-b890-9a90abe37805 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:24.124341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb861f62-b88a-47ad-bb48-27bd460f6471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:24.124842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf6762-a55c-4bf4-b0c4-e4a67e4cf5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:24.125336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb861f62-b88a-47ad-bb48-27bd460f6471 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:24.125517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf6762-a55c-4bf4-b0c4-e4a67e4cf5ef admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:24.125738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf6762-a55c-4bf4-b0c4-e4a67e4cf5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:24.125903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb861f62-b88a-47ad-bb48-27bd460f6471 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:24.126068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf6762-a55c-4bf4-b0c4-e4a67e4cf5ef admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:24.126442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb861f62-b88a-47ad-bb48-27bd460f6471 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:24.126670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf6762-a55c-4bf4-b0c4-e4a67e4cf5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:24.127461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb861f62-b88a-47ad-bb48-27bd460f6471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:24.128997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf6762-a55c-4bf4-b0c4-e4a67e4cf5ef admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:24.130558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb861f62-b88a-47ad-bb48-27bd460f6471 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:24.348882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 770/1581] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:24 2022] POST /identity/v3/users => generated 285 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:24.349940 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf6762-a55c-4bf4-b0c4-e4a67e4cf5ef admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.361893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf6762-a55c-4bf4-b0c4-e4a67e4cf5ef admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:24.368897 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3db232-b20f-4395-a583-0ae4db43659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:24.369933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3db232-b20f-4395-a583-0ae4db43659b admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:24.370393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3db232-b20f-4395-a583-0ae4db43659b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:24.370867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3db232-b20f-4395-a583-0ae4db43659b admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:24.372058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3db232-b20f-4395-a583-0ae4db43659b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:24.375137 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3db232-b20f-4395-a583-0ae4db43659b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:24.396062 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 812/1582] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:24 2022] POST /identity/v3/projects => generated 383 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:24.396062 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb861f62-b88a-47ad-bb48-27bd460f6471 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.397627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 771/1583] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:24.399764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3db232-b20f-4395-a583-0ae4db43659b admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.415358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:24.417703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3db232-b20f-4395-a583-0ae4db43659b admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:24.490598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:24.494214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb861f62-b88a-47ad-bb48-27bd460f6471 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:24.503779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d45476-d073-4aca-9dff-5c52f35111d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:24.504124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109ca8c-f132-4fc0-b8cf-3cbc17415c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:24.504795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109ca8c-f132-4fc0-b8cf-3cbc17415c03 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:24.505027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d45476-d073-4aca-9dff-5c52f35111d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:24.505213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109ca8c-f132-4fc0-b8cf-3cbc17415c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:24.505382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109ca8c-f132-4fc0-b8cf-3cbc17415c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:24.505517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d45476-d073-4aca-9dff-5c52f35111d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:24.505982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109ca8c-f132-4fc0-b8cf-3cbc17415c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:24.506127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d45476-d073-4aca-9dff-5c52f35111d5 admin admin] PATH_INFO: `/v3/projects/b32ff677e86d4be0b49869e579385a21/users/64e03bf24bb44add8adea800932e2f75/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:24.508703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109ca8c-f132-4fc0-b8cf-3cbc17415c03 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:24.536056 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d45476-d073-4aca-9dff-5c52f35111d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32ff677e86d4be0b49869e579385a21, user_id=64e03bf24bb44add8adea800932e2f75, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:24.541811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d45476-d073-4aca-9dff-5c52f35111d5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:24.619842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 772/1584] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:24 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:24.620986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109ca8c-f132-4fc0-b8cf-3cbc17415c03 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.651769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 813/1585] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:24 2022] PUT /identity/v3/projects/b32ff677e86d4be0b49869e579385a21/users/64e03bf24bb44add8adea800932e2f75/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:24.653248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d45476-d073-4aca-9dff-5c52f35111d5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.707439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109ca8c-f132-4fc0-b8cf-3cbc17415c03 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:24.738036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d45476-d073-4aca-9dff-5c52f35111d5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:24.743982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bfcbf-e867-48df-9c6d-97ea4fe26017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:24.744670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bfcbf-e867-48df-9c6d-97ea4fe26017 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:24.744953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bfcbf-e867-48df-9c6d-97ea4fe26017 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:24.745222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bfcbf-e867-48df-9c6d-97ea4fe26017 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:24.745878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bfcbf-e867-48df-9c6d-97ea4fe26017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:24.747896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bfcbf-e867-48df-9c6d-97ea4fe26017 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:24.798117 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbd502-5614-49d9-815a-d15b0eefc053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:24.798839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbd502-5614-49d9-815a-d15b0eefc053 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:24.799188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbd502-5614-49d9-815a-d15b0eefc053 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:24.799540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbd502-5614-49d9-815a-d15b0eefc053 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:24.800392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbd502-5614-49d9-815a-d15b0eefc053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:24.804585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbd502-5614-49d9-815a-d15b0eefc053 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:24.916002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 814/1586] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:24 2022] POST /identity/v3/users => generated 294 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:24.916820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bfcbf-e867-48df-9c6d-97ea4fe26017 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.927777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bfcbf-e867-48df-9c6d-97ea4fe26017 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:24.933441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441b20f-b7bc-4e37-acb9-7a01f92ec9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:24.934076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b20f-b7bc-4e37-acb9-7a01f92ec9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:24.934362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b20f-b7bc-4e37-acb9-7a01f92ec9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:24.934639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441b20f-b7bc-4e37-acb9-7a01f92ec9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:24.935312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441b20f-b7bc-4e37-acb9-7a01f92ec9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:24.937117 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441b20f-b7bc-4e37-acb9-7a01f92ec9e5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:24.950523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 815/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:24.951452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441b20f-b7bc-4e37-acb9-7a01f92ec9e5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.962630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441b20f-b7bc-4e37-acb9-7a01f92ec9e5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:24.968078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8624d75-77ec-410f-a3e9-30516742a78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:24.968731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8624d75-77ec-410f-a3e9-30516742a78f admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:24.969012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8624d75-77ec-410f-a3e9-30516742a78f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:24.969317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8624d75-77ec-410f-a3e9-30516742a78f admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:24.969958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8624d75-77ec-410f-a3e9-30516742a78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:24.971613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8624d75-77ec-410f-a3e9-30516742a78f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:24.983738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 816/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:24.989184 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8624d75-77ec-410f-a3e9-30516742a78f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.989452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 773/1589] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:24 2022] POST /identity/v3/users => generated 287 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:24.989452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbd502-5614-49d9-815a-d15b0eefc053 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:24.999525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8624d75-77ec-410f-a3e9-30516742a78f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.004698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbd502-5614-49d9-815a-d15b0eefc053 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.004951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3eadbe-f6ad-4287-b871-c793b82d61d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.005460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eadbe-f6ad-4287-b871-c793b82d61d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.005790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eadbe-f6ad-4287-b871-c793b82d61d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.006091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eadbe-f6ad-4287-b871-c793b82d61d9 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.006993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eadbe-f6ad-4287-b871-c793b82d61d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.009119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eadbe-f6ad-4287-b871-c793b82d61d9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.009528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b1b59-1575-48cd-9ec5-b518c1e23a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.011059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b1b59-1575-48cd-9ec5-b518c1e23a51 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.011444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b1b59-1575-48cd-9ec5-b518c1e23a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.011880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b1b59-1575-48cd-9ec5-b518c1e23a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.012935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b1b59-1575-48cd-9ec5-b518c1e23a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.015359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b1b59-1575-48cd-9ec5-b518c1e23a51 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.028652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 774/1590] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:25.029655 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b1b59-1575-48cd-9ec5-b518c1e23a51 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.045762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b1b59-1575-48cd-9ec5-b518c1e23a51 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.047327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944aaaaa-aeb3-4974-b1bf-ec8bcacd22ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.047960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944aaaaa-aeb3-4974-b1bf-ec8bcacd22ea admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.048377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944aaaaa-aeb3-4974-b1bf-ec8bcacd22ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.048660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944aaaaa-aeb3-4974-b1bf-ec8bcacd22ea admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.049389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944aaaaa-aeb3-4974-b1bf-ec8bcacd22ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.051469 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944aaaaa-aeb3-4974-b1bf-ec8bcacd22ea admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.133841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 817/1591] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:24 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:25.136055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2483d-f5cc-4956-8c34-e176344eef93 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.136601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2483d-f5cc-4956-8c34-e176344eef93 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.137088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2483d-f5cc-4956-8c34-e176344eef93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.158497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-bdf2483d-f5cc-4956-8c34-e176344eef93 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:25.159752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-bdf2483d-f5cc-4956-8c34-e176344eef93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:25.168696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 818/1592] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:25.169514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2483d-f5cc-4956-8c34-e176344eef93 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.184296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2483d-f5cc-4956-8c34-e176344eef93 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.185827 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 775/1593] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:25 2022] POST /identity/v3/users => generated 299 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:25.187156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b332e-690b-4787-aa2e-014190ab536f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.187345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944aaaaa-aeb3-4974-b1bf-ec8bcacd22ea admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.188132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b332e-690b-4787-aa2e-014190ab536f admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.188604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b332e-690b-4787-aa2e-014190ab536f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.189046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b332e-690b-4787-aa2e-014190ab536f admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.189869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b332e-690b-4787-aa2e-014190ab536f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.191695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b332e-690b-4787-aa2e-014190ab536f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.202492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944aaaaa-aeb3-4974-b1bf-ec8bcacd22ea admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.203822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 819/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:25.204128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b332e-690b-4787-aa2e-014190ab536f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.212398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5497803-0d2a-4f23-810a-1d3742c313bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.213276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5497803-0d2a-4f23-810a-1d3742c313bf admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.213699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5497803-0d2a-4f23-810a-1d3742c313bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.214070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5497803-0d2a-4f23-810a-1d3742c313bf admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.214890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5497803-0d2a-4f23-810a-1d3742c313bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.215465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b332e-690b-4787-aa2e-014190ab536f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.217103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5497803-0d2a-4f23-810a-1d3742c313bf admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.220640 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e64ac-6643-4564-9e11-8ace8284cc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.221401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e64ac-6643-4564-9e11-8ace8284cc0d admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.221692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e64ac-6643-4564-9e11-8ace8284cc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.222028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e64ac-6643-4564-9e11-8ace8284cc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.222601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e64ac-6643-4564-9e11-8ace8284cc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.224309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e64ac-6643-4564-9e11-8ace8284cc0d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.232212 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 776/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:25.233692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5497803-0d2a-4f23-810a-1d3742c313bf admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.236772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 820/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:25.237641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e64ac-6643-4564-9e11-8ace8284cc0d admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.249494 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e64ac-6643-4564-9e11-8ace8284cc0d admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.251358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5497803-0d2a-4f23-810a-1d3742c313bf admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.256931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d110175-95c2-4909-a1df-0f3a302aef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.257737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d110175-95c2-4909-a1df-0f3a302aef40 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.258078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d110175-95c2-4909-a1df-0f3a302aef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.258471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d110175-95c2-4909-a1df-0f3a302aef40 admin admin] PATH_INFO: `/v3/projects/4eb043e789a6467db5465a0107f41076/users/fd040e13fd6c4b0dbea6a65932f84cb2/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.260262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901845e-3783-47fb-aacb-8f55fa7d7d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.261856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901845e-3783-47fb-aacb-8f55fa7d7d6f admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.262759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901845e-3783-47fb-aacb-8f55fa7d7d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.263382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901845e-3783-47fb-aacb-8f55fa7d7d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.265208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901845e-3783-47fb-aacb-8f55fa7d7d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.268400 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901845e-3783-47fb-aacb-8f55fa7d7d6f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.278815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:25.285969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 777/1597] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:25.285969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901845e-3783-47fb-aacb-8f55fa7d7d6f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.286380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d110175-95c2-4909-a1df-0f3a302aef40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eb043e789a6467db5465a0107f41076, user_id=fd040e13fd6c4b0dbea6a65932f84cb2, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.291013 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d110175-95c2-4909-a1df-0f3a302aef40 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.294630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901845e-3783-47fb-aacb-8f55fa7d7d6f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.299960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21f797-fb47-44ec-a6fc-455d0c256fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.300625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21f797-fb47-44ec-a6fc-455d0c256fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.300931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21f797-fb47-44ec-a6fc-455d0c256fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.301208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21f797-fb47-44ec-a6fc-455d0c256fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.301855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21f797-fb47-44ec-a6fc-455d0c256fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.303505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21f797-fb47-44ec-a6fc-455d0c256fd2 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.315939 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 778/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:25.316878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21f797-fb47-44ec-a6fc-455d0c256fd2 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.319061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2a21f797-fb47-44ec-a6fc-455d0c256fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:25.335690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:25.390066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 821/1599] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:25 2022] PUT /identity/v3/projects/4eb043e789a6467db5465a0107f41076/users/fd040e13fd6c4b0dbea6a65932f84cb2/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:25.391479 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d110175-95c2-4909-a1df-0f3a302aef40 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.395230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21f797-fb47-44ec-a6fc-455d0c256fd2 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.459504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89d46d-c558-4988-8800-aa964d32a54e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.460578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89d46d-c558-4988-8800-aa964d32a54e service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.460857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89d46d-c558-4988-8800-aa964d32a54e service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.461187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89d46d-c558-4988-8800-aa964d32a54e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.492730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d110175-95c2-4909-a1df-0f3a302aef40 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.497978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ba811-fdbd-4a88-acd1-2fa9966280d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.498632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ba811-fdbd-4a88-acd1-2fa9966280d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.498908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ba811-fdbd-4a88-acd1-2fa9966280d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.499181 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ba811-fdbd-4a88-acd1-2fa9966280d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.499835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ba811-fdbd-4a88-acd1-2fa9966280d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.501542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ba811-fdbd-4a88-acd1-2fa9966280d7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.517786 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 822/1600] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:25.519453 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ba811-fdbd-4a88-acd1-2fa9966280d7 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.537945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ba811-fdbd-4a88-acd1-2fa9966280d7 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.546820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d3aef-e693-4169-94ca-a933830780be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.547845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d3aef-e693-4169-94ca-a933830780be admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.548394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d3aef-e693-4169-94ca-a933830780be admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.548910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d3aef-e693-4169-94ca-a933830780be admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.549981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d3aef-e693-4169-94ca-a933830780be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.553173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d3aef-e693-4169-94ca-a933830780be admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.558345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89d46d-c558-4988-8800-aa964d32a54e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.560158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89d46d-c558-4988-8800-aa964d32a54e service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.569288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 823/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:25.570172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d3aef-e693-4169-94ca-a933830780be admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.571997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 779/1602] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:25.572667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89d46d-c558-4988-8800-aa964d32a54e service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.581945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d3aef-e693-4169-94ca-a933830780be admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.586163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89d46d-c558-4988-8800-aa964d32a54e service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.587942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222a97-50bf-4171-be09-d2a7feebf8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.588662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222a97-50bf-4171-be09-d2a7feebf8e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.589040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222a97-50bf-4171-be09-d2a7feebf8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.589450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222a97-50bf-4171-be09-d2a7feebf8e2 admin admin] PATH_INFO: `/v3/projects/052474df3ff8460ea78de214416439ec/users/4fa4c7991f3247dc91b03cd61d303c09/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.591863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6cdc4-7b2f-4a94-ade4-581b326bc852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.592653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6cdc4-7b2f-4a94-ade4-581b326bc852 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.592926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6cdc4-7b2f-4a94-ade4-581b326bc852 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.593185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6cdc4-7b2f-4a94-ade4-581b326bc852 admin admin] PATH_INFO: `/v3/projects/20f2e63652624b738fbc342edeedeb62/users/210399c956734c76bb78137afd48dafd/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.609311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222a97-50bf-4171-be09-d2a7feebf8e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=052474df3ff8460ea78de214416439ec, user_id=4fa4c7991f3247dc91b03cd61d303c09, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.612213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222a97-50bf-4171-be09-d2a7feebf8e2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.619169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6cdc4-7b2f-4a94-ade4-581b326bc852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20f2e63652624b738fbc342edeedeb62, user_id=210399c956734c76bb78137afd48dafd, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.620889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WARNING py.warnings [None req-08f6cdc4-7b2f-4a94-ade4-581b326bc852 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:02:25.620889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: warnings.warn(msg) Oct 04 12:02:25.620889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: Oct 04 12:02:25.624302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6cdc4-7b2f-4a94-ade4-581b326bc852 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.746647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 824/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:25 2022] PUT /identity/v3/projects/052474df3ff8460ea78de214416439ec/users/4fa4c7991f3247dc91b03cd61d303c09/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:25.746647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222a97-50bf-4171-be09-d2a7feebf8e2 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.769988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 780/1604] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:25 2022] PUT /identity/v3/projects/20f2e63652624b738fbc342edeedeb62/users/210399c956734c76bb78137afd48dafd/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:25.771467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6cdc4-7b2f-4a94-ade4-581b326bc852 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.876746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222a97-50bf-4171-be09-d2a7feebf8e2 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.895428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:25.910759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6cdc4-7b2f-4a94-ade4-581b326bc852 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:25.916324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb03675-20a1-4cdc-9a86-1944811ad224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.916947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb03675-20a1-4cdc-9a86-1944811ad224 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.917224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb03675-20a1-4cdc-9a86-1944811ad224 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.917499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb03675-20a1-4cdc-9a86-1944811ad224 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.918158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb03675-20a1-4cdc-9a86-1944811ad224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.919848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb03675-20a1-4cdc-9a86-1944811ad224 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.935753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 781/1605] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:25.938476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5537d-c243-4a42-acd4-7f5b9464833f None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.941652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5537d-c243-4a42-acd4-7f5b9464833f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.941652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5537d-c243-4a42-acd4-7f5b9464833f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.958246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a1ea1-f478-42f2-8d1c-427d401dcb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:25.959008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a1ea1-f478-42f2-8d1c-427d401dcb68 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:25.959457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a1ea1-f478-42f2-8d1c-427d401dcb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:25.960081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a1ea1-f478-42f2-8d1c-427d401dcb68 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:25.960990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a1ea1-f478-42f2-8d1c-427d401dcb68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:25.963328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a1ea1-f478-42f2-8d1c-427d401dcb68 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:25.965557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:25.968927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d7b5537d-c243-4a42-acd4-7f5b9464833f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:25.970786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d7b5537d-c243-4a42-acd4-7f5b9464833f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:25.978205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 825/1606] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:25.979063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a1ea1-f478-42f2-8d1c-427d401dcb68 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:25.981420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 782/1607] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:25.982484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5537d-c243-4a42-acd4-7f5b9464833f None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:26.002831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a1ea1-f478-42f2-8d1c-427d401dcb68 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:26.003091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5537d-c243-4a42-acd4-7f5b9464833f None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:26.003530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cddb5-b552-406d-8568-58b7136460fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:26.003736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d06489-0361-4bc2-8d10-b4b22a8a7308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:26.004290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cddb5-b552-406d-8568-58b7136460fb admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:26.004522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d06489-0361-4bc2-8d10-b4b22a8a7308 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:26.004801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cddb5-b552-406d-8568-58b7136460fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:26.004975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d06489-0361-4bc2-8d10-b4b22a8a7308 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:26.005275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d06489-0361-4bc2-8d10-b4b22a8a7308 admin admin] PATH_INFO: `/v3/projects/05221a6000364bfea83e82d2eba93b1a/users/f1e591a6065a41a78a74591fcc9ab8e3/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:26.005475 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cddb5-b552-406d-8568-58b7136460fb admin admin] PATH_INFO: `/v3/projects/9741380a634142c1b9b4065cdf7a15f9/users/7dabf36d8e36481da167d8e29ebbe7a5/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:26.031239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cddb5-b552-406d-8568-58b7136460fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9741380a634142c1b9b4065cdf7a15f9, user_id=7dabf36d8e36481da167d8e29ebbe7a5, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:26.031621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d06489-0361-4bc2-8d10-b4b22a8a7308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05221a6000364bfea83e82d2eba93b1a, user_id=f1e591a6065a41a78a74591fcc9ab8e3, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:26.044628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cddb5-b552-406d-8568-58b7136460fb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:26.044871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d06489-0361-4bc2-8d10-b4b22a8a7308 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:26.306636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 826/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:25 2022] PUT /identity/v3/projects/9741380a634142c1b9b4065cdf7a15f9/users/7dabf36d8e36481da167d8e29ebbe7a5/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:26.306636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10a18-c0fd-4401-8755-995e40758f3d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:26.306636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10a18-c0fd-4401-8755-995e40758f3d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:26.306636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10a18-c0fd-4401-8755-995e40758f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:26.333826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2ce10a18-c0fd-4401-8755-995e40758f3d None None] MFA Rules not processed for user `4fa4c7991f3247dc91b03cd61d303c09`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:26.337272 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2ce10a18-c0fd-4401-8755-995e40758f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:26.352881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 783/1609] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:25 2022] PUT /identity/v3/projects/05221a6000364bfea83e82d2eba93b1a/users/f1e591a6065a41a78a74591fcc9ab8e3/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:26.372066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a025491-23eb-4f7d-bea0-7c1c3a5b8581 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:26.373341 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a025491-23eb-4f7d-bea0-7c1c3a5b8581 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:26.374278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a025491-23eb-4f7d-bea0-7c1c3a5b8581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:26.421959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-9a025491-23eb-4f7d-bea0-7c1c3a5b8581 None None] MFA Rules not processed for user `210399c956734c76bb78137afd48dafd`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:26.423353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-9a025491-23eb-4f7d-bea0-7c1c3a5b8581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:26.449334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 827/1610] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:26.451611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce10a18-c0fd-4401-8755-995e40758f3d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:26.513395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 784/1611] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:26.515039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a025491-23eb-4f7d-bea0-7c1c3a5b8581 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:26.542054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce10a18-c0fd-4401-8755-995e40758f3d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:26.622718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e638c-88e6-4b7b-8e5d-814a180e20a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:26.623411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e638c-88e6-4b7b-8e5d-814a180e20a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:26.623692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e638c-88e6-4b7b-8e5d-814a180e20a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:26.623970 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e638c-88e6-4b7b-8e5d-814a180e20a7 admin admin] PATH_INFO: `/v3/projects/f021c9955236461abc441e2a31421b8b/users/0e8d03fe51534f8e867d0f83c3db2d36/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:26.637699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a025491-23eb-4f7d-bea0-7c1c3a5b8581 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:26.644873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99ab29-23e5-46cd-b443-ca47627adecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:26.645955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ab29-23e5-46cd-b443-ca47627adecf admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:26.646358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ab29-23e5-46cd-b443-ca47627adecf admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:26.646732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99ab29-23e5-46cd-b443-ca47627adecf admin admin] PATH_INFO: `/v3/projects/57c344f95027402e8598ed54ad453c7f/users/513b7df0970e436ca894a31ae4321b5e/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:26.646881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:26.652711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e638c-88e6-4b7b-8e5d-814a180e20a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f021c9955236461abc441e2a31421b8b, user_id=0e8d03fe51534f8e867d0f83c3db2d36, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:26.657497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e638c-88e6-4b7b-8e5d-814a180e20a7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:26.673297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99ab29-23e5-46cd-b443-ca47627adecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c344f95027402e8598ed54ad453c7f, user_id=513b7df0970e436ca894a31ae4321b5e, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:26.680216 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99ab29-23e5-46cd-b443-ca47627adecf admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:26.917779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 828/1612] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:26 2022] PUT /identity/v3/projects/f021c9955236461abc441e2a31421b8b/users/0e8d03fe51534f8e867d0f83c3db2d36/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:26.919239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e638c-88e6-4b7b-8e5d-814a180e20a7 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:26.960601 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 785/1613] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:26 2022] PUT /identity/v3/projects/57c344f95027402e8598ed54ad453c7f/users/513b7df0970e436ca894a31ae4321b5e/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:26.960601 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99ab29-23e5-46cd-b443-ca47627adecf admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:27.023597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e638c-88e6-4b7b-8e5d-814a180e20a7 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:27.027577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:27.058906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99ab29-23e5-46cd-b443-ca47627adecf admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:27.064752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37d92e-ebbf-4bf2-936d-db4d69a10ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:27.065445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37d92e-ebbf-4bf2-936d-db4d69a10ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.065803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37d92e-ebbf-4bf2-936d-db4d69a10ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.066246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37d92e-ebbf-4bf2-936d-db4d69a10ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.067133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37d92e-ebbf-4bf2-936d-db4d69a10ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:27.069761 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37d92e-ebbf-4bf2-936d-db4d69a10ed4 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:27.085558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 786/1614] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:27.088713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c4c89-0fbc-4523-8c28-b67980ad53a3 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.088713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c4c89-0fbc-4523-8c28-b67980ad53a3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.088713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c4c89-0fbc-4523-8c28-b67980ad53a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.093206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f1f04-b600-4f4a-bdf4-28de4aaef959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:27.093872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f1f04-b600-4f4a-bdf4-28de4aaef959 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.094160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f1f04-b600-4f4a-bdf4-28de4aaef959 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.094457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f1f04-b600-4f4a-bdf4-28de4aaef959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.104655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f1f04-b600-4f4a-bdf4-28de4aaef959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:27.106106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f1f04-b600-4f4a-bdf4-28de4aaef959 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:27.117468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 829/1615] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:27.118449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f1f04-b600-4f4a-bdf4-28de4aaef959 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:27.120738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7d6c4c89-0fbc-4523-8c28-b67980ad53a3 None None] MFA Rules not processed for user `f1e591a6065a41a78a74591fcc9ab8e3`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:27.122047 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7d6c4c89-0fbc-4523-8c28-b67980ad53a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:27.129995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f1f04-b600-4f4a-bdf4-28de4aaef959 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:27.139221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d32cb-5f4d-4f9d-830e-20ed22758a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:27.140338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d32cb-5f4d-4f9d-830e-20ed22758a93 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.140972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d32cb-5f4d-4f9d-830e-20ed22758a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.141639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d32cb-5f4d-4f9d-830e-20ed22758a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.142526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d32cb-5f4d-4f9d-830e-20ed22758a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:27.145528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d32cb-5f4d-4f9d-830e-20ed22758a93 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:27.220926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 787/1616] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:27.222296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da288be4-3857-42fc-9506-5d93f13a7b06 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.222577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da288be4-3857-42fc-9506-5d93f13a7b06 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.222883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da288be4-3857-42fc-9506-5d93f13a7b06 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.224574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 788/1617] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 4 12:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:02:27.225962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c94d4f-4c2d-43ca-8f53-b674ba32771f None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.226304 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c94d4f-4c2d-43ca-8f53-b674ba32771f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.226679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c94d4f-4c2d-43ca-8f53-b674ba32771f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.250283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-30c94d4f-4c2d-43ca-8f53-b674ba32771f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:27.251901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-30c94d4f-4c2d-43ca-8f53-b674ba32771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:27.261799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 789/1618] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:27.263337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200a013-6d85-4e61-ad43-63794221dff6 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.263731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200a013-6d85-4e61-ad43-63794221dff6 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.264100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200a013-6d85-4e61-ad43-63794221dff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.293403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 830/1619] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:27 2022] POST /identity/v3/projects => generated 405 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:27.297588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d32cb-5f4d-4f9d-830e-20ed22758a93 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:27.317347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a200a013-6d85-4e61-ad43-63794221dff6 None None] MFA Rules not processed for user `513b7df0970e436ca894a31ae4321b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:27.317347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a200a013-6d85-4e61-ad43-63794221dff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:27.400858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d32cb-5f4d-4f9d-830e-20ed22758a93 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:27.407777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ff284-9250-4226-811f-4d562419f78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:27.408603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff284-9250-4226-811f-4d562419f78b admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.408880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff284-9250-4226-811f-4d562419f78b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.410136 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff284-9250-4226-811f-4d562419f78b admin admin] PATH_INFO: `/v3/projects/9741380a634142c1b9b4065cdf7a15f9/users/7dabf36d8e36481da167d8e29ebbe7a5/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.413726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ff284-9250-4226-811f-4d562419f78b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9741380a634142c1b9b4065cdf7a15f9, user_id=7dabf36d8e36481da167d8e29ebbe7a5, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:27.416376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 790/1620] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:27.416946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ff284-9250-4226-811f-4d562419f78b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:27.418449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200a013-6d85-4e61-ad43-63794221dff6 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:27.431886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 831/1621] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:27 2022] PUT /identity/v3/projects/9741380a634142c1b9b4065cdf7a15f9/users/7dabf36d8e36481da167d8e29ebbe7a5/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:27.433973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe90b3-366d-48a5-bd32-1bd71e11f175 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.434268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe90b3-366d-48a5-bd32-1bd71e11f175 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.434543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe90b3-366d-48a5-bd32-1bd71e11f175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.442353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200a013-6d85-4e61-ad43-63794221dff6 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:27.525067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-ffbe90b3-366d-48a5-bd32-1bd71e11f175 None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:27.526664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ffbe90b3-366d-48a5-bd32-1bd71e11f175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:27.532882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:27.538577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17b8b9-82cf-4c79-adc3-36a9f61db2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:27.539996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17b8b9-82cf-4c79-adc3-36a9f61db2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.540417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17b8b9-82cf-4c79-adc3-36a9f61db2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.540985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17b8b9-82cf-4c79-adc3-36a9f61db2f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.541227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17b8b9-82cf-4c79-adc3-36a9f61db2f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:27.543214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17b8b9-82cf-4c79-adc3-36a9f61db2f0 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:27.573968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:27.656885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 832/1622] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 4 12:02:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:27.657783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe90b3-366d-48a5-bd32-1bd71e11f175 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:27.724365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe90b3-366d-48a5-bd32-1bd71e11f175 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:27.728399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867436a-4666-4264-a759-84cd6bb6b9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:27.729043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867436a-4666-4264-a759-84cd6bb6b9ca service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.729323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867436a-4666-4264-a759-84cd6bb6b9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.729597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867436a-4666-4264-a759-84cd6bb6b9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.740594 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867436a-4666-4264-a759-84cd6bb6b9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:27.745351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867436a-4666-4264-a759-84cd6bb6b9ca service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:27.760706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 791/1623] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:27 2022] POST /identity/v3/projects => generated 387 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:27.760706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17b8b9-82cf-4c79-adc3-36a9f61db2f0 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:27.767802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 833/1624] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:27.769500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867436a-4666-4264-a759-84cd6bb6b9ca service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:27.786489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867436a-4666-4264-a759-84cd6bb6b9ca service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:27.892212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef17d92-a9b4-4b15-a451-54629a8ec01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:27.893327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17d92-a9b4-4b15-a451-54629a8ec01b admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.893796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17d92-a9b4-4b15-a451-54629a8ec01b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.894434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17d92-a9b4-4b15-a451-54629a8ec01b admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.898007 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef17d92-a9b4-4b15-a451-54629a8ec01b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:27.903235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef17d92-a9b4-4b15-a451-54629a8ec01b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:27.923206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17b8b9-82cf-4c79-adc3-36a9f61db2f0 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:27.933931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b7557-2ee8-4af8-b409-0bcfa5486fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:27.934536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7557-2ee8-4af8-b409-0bcfa5486fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:27.935119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7557-2ee8-4af8-b409-0bcfa5486fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:27.936657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7557-2ee8-4af8-b409-0bcfa5486fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:27.938043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b7557-2ee8-4af8-b409-0bcfa5486fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:27.941843 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b7557-2ee8-4af8-b409-0bcfa5486fd6 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:28.083598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 834/1625] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:27 2022] POST /identity/v3/projects => generated 371 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:28.085665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b730a6-098d-4c19-9ba3-68916074263f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:28.086273 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b730a6-098d-4c19-9ba3-68916074263f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:28.086668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b730a6-098d-4c19-9ba3-68916074263f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:28.138430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 792/1626] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:27 2022] POST /identity/v3/users => generated 309 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:28.138430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b7557-2ee8-4af8-b409-0bcfa5486fd6 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:28.143778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-89b730a6-098d-4c19-9ba3-68916074263f None None] MFA Rules not processed for user `7dabf36d8e36481da167d8e29ebbe7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:28.146026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-89b730a6-098d-4c19-9ba3-68916074263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:28.488359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:28.488806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b7557-2ee8-4af8-b409-0bcfa5486fd6 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:28.488958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 835/1627] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:28.488958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b730a6-098d-4c19-9ba3-68916074263f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:28.501830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:28.537769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e253da8-ae1f-4ae1-885a-61e3bcb92f26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:28.538823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e253da8-ae1f-4ae1-885a-61e3bcb92f26 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:28.539198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e253da8-ae1f-4ae1-885a-61e3bcb92f26 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:28.539511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e253da8-ae1f-4ae1-885a-61e3bcb92f26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:28.563280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b730a6-098d-4c19-9ba3-68916074263f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:28.568913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b76a14-214b-4453-8bfe-7a41a1ce1eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:28.569640 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b76a14-214b-4453-8bfe-7a41a1ce1eeb admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:28.571190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b76a14-214b-4453-8bfe-7a41a1ce1eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:28.571190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b76a14-214b-4453-8bfe-7a41a1ce1eeb admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:28.571190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b76a14-214b-4453-8bfe-7a41a1ce1eeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:28.572932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b76a14-214b-4453-8bfe-7a41a1ce1eeb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:28.628989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e253da8-ae1f-4ae1-885a-61e3bcb92f26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:28.630933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e253da8-ae1f-4ae1-885a-61e3bcb92f26 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:28.644872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 793/1628] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:28.646023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e253da8-ae1f-4ae1-885a-61e3bcb92f26 service nova] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:28.661059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e253da8-ae1f-4ae1-885a-61e3bcb92f26 service nova] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:28.671290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375f60-93de-41e5-9e79-593b3e5d496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:28.672139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375f60-93de-41e5-9e79-593b3e5d496b admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:28.672722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375f60-93de-41e5-9e79-593b3e5d496b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:28.673230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375f60-93de-41e5-9e79-593b3e5d496b admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:28.674841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27375f60-93de-41e5-9e79-593b3e5d496b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:28.678260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27375f60-93de-41e5-9e79-593b3e5d496b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:28.734792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 836/1629] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:28 2022] POST /identity/v3/users => generated 300 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:28.737115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b76a14-214b-4453-8bfe-7a41a1ce1eeb admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:28.759671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b76a14-214b-4453-8bfe-7a41a1ce1eeb admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:28.773138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e07ec-f58c-4a39-a8af-6e8295e4d73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:28.774326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e07ec-f58c-4a39-a8af-6e8295e4d73d admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:28.774966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e07ec-f58c-4a39-a8af-6e8295e4d73d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:28.775542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e07ec-f58c-4a39-a8af-6e8295e4d73d admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:28.776978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e07ec-f58c-4a39-a8af-6e8295e4d73d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:28.780505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e07ec-f58c-4a39-a8af-6e8295e4d73d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:28.845440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 837/1630] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:28.846693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e07ec-f58c-4a39-a8af-6e8295e4d73d admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:28.867794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 794/1631] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:28 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:28.867794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375f60-93de-41e5-9e79-593b3e5d496b admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:28.922924 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375f60-93de-41e5-9e79-593b3e5d496b admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:28.953667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112d3aa-e2fd-421c-9bf7-3344915789e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:28.954748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112d3aa-e2fd-421c-9bf7-3344915789e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:28.955246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112d3aa-e2fd-421c-9bf7-3344915789e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:28.955989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112d3aa-e2fd-421c-9bf7-3344915789e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:28.956958 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112d3aa-e2fd-421c-9bf7-3344915789e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:28.960985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112d3aa-e2fd-421c-9bf7-3344915789e8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:28.980976 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e07ec-f58c-4a39-a8af-6e8295e4d73d admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:28.988251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0aa7-71ee-46f1-99aa-35c4cf6ea803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:28.989124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0aa7-71ee-46f1-99aa-35c4cf6ea803 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:28.989524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 795/1632] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:28.990587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0aa7-71ee-46f1-99aa-35c4cf6ea803 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:28.991016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112d3aa-e2fd-421c-9bf7-3344915789e8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:28.991238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0aa7-71ee-46f1-99aa-35c4cf6ea803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:29.018529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d0aa7-71ee-46f1-99aa-35c4cf6ea803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:29.024459 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d0aa7-71ee-46f1-99aa-35c4cf6ea803 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:29.027280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112d3aa-e2fd-421c-9bf7-3344915789e8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:29.040028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb2246-ca0e-4b7e-a8bb-04c1928dfdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:29.043683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb2246-ca0e-4b7e-a8bb-04c1928dfdc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:29.044066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb2246-ca0e-4b7e-a8bb-04c1928dfdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:29.044379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb2246-ca0e-4b7e-a8bb-04c1928dfdc8 admin admin] PATH_INFO: `/v3/projects/586cb24019004587b71d7817f45e7e56/users/affac764cb964a06ae30275fabc0edfd/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:29.060215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 838/1633] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:29.074361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0aa7-71ee-46f1-99aa-35c4cf6ea803 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:29.112847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0aa7-71ee-46f1-99aa-35c4cf6ea803 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:29.118545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb2246-ca0e-4b7e-a8bb-04c1928dfdc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=586cb24019004587b71d7817f45e7e56, user_id=affac764cb964a06ae30275fabc0edfd, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:29.125275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb2246-ca0e-4b7e-a8bb-04c1928dfdc8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:29.137244 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f3322-f281-49cc-9183-248ad79458ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:29.138203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f3322-f281-49cc-9183-248ad79458ba admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:29.138490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f3322-f281-49cc-9183-248ad79458ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:29.138769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f3322-f281-49cc-9183-248ad79458ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:29.139465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f3322-f281-49cc-9183-248ad79458ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:29.141705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f3322-f281-49cc-9183-248ad79458ba admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:29.159305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 839/1634] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:29.159305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fb019-242b-4ac1-b6de-b450641210c1 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:29.159305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fb019-242b-4ac1-b6de-b450641210c1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:29.159305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fb019-242b-4ac1-b6de-b450641210c1 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:29.159305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 840/1635] 10.222.0.15 () {54 vars in 912 bytes} [Tue Oct 4 12:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:02:29.159305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4fb019-242b-4ac1-b6de-b450641210c1 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:29.204891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4fb019-242b-4ac1-b6de-b450641210c1 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:29.218851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc13238-d131-4d1c-ac18-366259ddb945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:29.219906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc13238-d131-4d1c-ac18-366259ddb945 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:29.220256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc13238-d131-4d1c-ac18-366259ddb945 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:29.220713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc13238-d131-4d1c-ac18-366259ddb945 admin admin] PATH_INFO: `/v3/projects/b575d8ed68ab4b5eb9bec3bfd204991b/users/86cde6620728449f92adacc51f67df10/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:29.241640 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc13238-d131-4d1c-ac18-366259ddb945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b575d8ed68ab4b5eb9bec3bfd204991b, user_id=86cde6620728449f92adacc51f67df10, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:29.244888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc13238-d131-4d1c-ac18-366259ddb945 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:29.271774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 796/1636] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:28 2022] PUT /identity/v3/projects/586cb24019004587b71d7817f45e7e56/users/affac764cb964a06ae30275fabc0edfd/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:29.272942 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb2246-ca0e-4b7e-a8bb-04c1928dfdc8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:29.356007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb2246-ca0e-4b7e-a8bb-04c1928dfdc8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:29.362465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:29.363539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:29.364046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:29.364576 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] PATH_INFO: `/v3/users/0f7fc9f080e94b7f904e020be896b5a3` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:29.366896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7fc9f080e94b7f904e020be896b5a3)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:29.368473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: WARNING py.warnings [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:02:29.368473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: warnings.warn(msg) Oct 04 12:02:29.368473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: Oct 04 12:02:29.370296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:29.384829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:29.393069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 841/1637] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:29 2022] PUT /identity/v3/projects/b575d8ed68ab4b5eb9bec3bfd204991b/users/86cde6620728449f92adacc51f67df10/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:29.394066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc13238-d131-4d1c-ac18-366259ddb945 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:29.518785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc13238-d131-4d1c-ac18-366259ddb945 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:29.525706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9ae03-0b16-4ed4-ab25-87d0645f5b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:29.526821 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9ae03-0b16-4ed4-ab25-87d0645f5b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:29.527232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9ae03-0b16-4ed4-ab25-87d0645f5b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:29.527589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9ae03-0b16-4ed4-ab25-87d0645f5b12 admin admin] PATH_INFO: `/v3/projects/1219f4d297154dbdb51c46f0a439d15d/users/87a2a0f44caf48469d7ca3d6fb78c86d/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:29.552136 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9ae03-0b16-4ed4-ab25-87d0645f5b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1219f4d297154dbdb51c46f0a439d15d, user_id=87a2a0f44caf48469d7ca3d6fb78c86d, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:29.557167 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9ae03-0b16-4ed4-ab25-87d0645f5b12 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:29.725797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 842/1638] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:29 2022] PUT /identity/v3/projects/1219f4d297154dbdb51c46f0a439d15d/users/87a2a0f44caf48469d7ca3d6fb78c86d/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:29.734706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336fc2f-77ba-48ed-ba4d-8e8f3d59a08e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:29.735118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336fc2f-77ba-48ed-ba4d-8e8f3d59a08e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:29.735609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336fc2f-77ba-48ed-ba4d-8e8f3d59a08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:29.759562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:29.766572 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e336fc2f-77ba-48ed-ba4d-8e8f3d59a08e None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:29.769052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e336fc2f-77ba-48ed-ba4d-8e8f3d59a08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:29.774507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7fc9f080e94b7f904e020be896b5a3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:29.814938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7fc9f080e94b7f904e020be896b5a3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:29.816553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7fc9f080e94b7f904e020be896b5a3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:29.943569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7fc9f080e94b7f904e020be896b5a3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:29.947671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 843/1639] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:02:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:29.949530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336fc2f-77ba-48ed-ba4d-8e8f3d59a08e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:29.952897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e336fc2f-77ba-48ed-ba4d-8e8f3d59a08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:29.992922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7fc9f080e94b7f904e020be896b5a3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:30.010328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 797/1640] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:29 2022] DELETE /identity/v3/users/0f7fc9f080e94b7f904e020be896b5a3 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:30.015551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:30.017489 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:30.088815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336fc2f-77ba-48ed-ba4d-8e8f3d59a08e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:30.165570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f606f7-ff05-4569-88de-d34332d0d3f1 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:30.174301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc509705-38e1-4d88-aa85-ebfdb5abfada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:30.175548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc509705-38e1-4d88-aa85-ebfdb5abfada admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:30.176007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc509705-38e1-4d88-aa85-ebfdb5abfada admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:30.176592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc509705-38e1-4d88-aa85-ebfdb5abfada admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:30.181386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc509705-38e1-4d88-aa85-ebfdb5abfada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:30.182861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc509705-38e1-4d88-aa85-ebfdb5abfada admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:30.206229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 798/1641] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:30.207946 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc509705-38e1-4d88-aa85-ebfdb5abfada admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:30.211912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cc509705-38e1-4d88-aa85-ebfdb5abfada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:30.240407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0a31a-3c10-4b61-b5f6-a1b5ad56437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:30.242196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a31a-3c10-4b61-b5f6-a1b5ad56437f admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:30.242795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a31a-3c10-4b61-b5f6-a1b5ad56437f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:30.244108 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a31a-3c10-4b61-b5f6-a1b5ad56437f admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:30.245503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0a31a-3c10-4b61-b5f6-a1b5ad56437f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:30.248972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0a31a-3c10-4b61-b5f6-a1b5ad56437f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:30.250271 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc509705-38e1-4d88-aa85-ebfdb5abfada admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:30.261120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd6fa2-d970-400c-a6b1-71ffc0234bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:30.262217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd6fa2-d970-400c-a6b1-71ffc0234bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:30.262833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd6fa2-d970-400c-a6b1-71ffc0234bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:30.263330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd6fa2-d970-400c-a6b1-71ffc0234bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:30.265915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd6fa2-d970-400c-a6b1-71ffc0234bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:30.269953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd6fa2-d970-400c-a6b1-71ffc0234bb9 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:30.277604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 844/1642] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:30.279274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0a31a-3c10-4b61-b5f6-a1b5ad56437f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:30.289094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2db0a31a-3c10-4b61-b5f6-a1b5ad56437f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:30.310631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 799/1643] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:30.314788 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69453-712b-4e27-9894-64c9c222f8ea None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:30.315788 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69453-712b-4e27-9894-64c9c222f8ea None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:30.317043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69453-712b-4e27-9894-64c9c222f8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:30.362963 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-67d69453-712b-4e27-9894-64c9c222f8ea None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:30.365100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-67d69453-712b-4e27-9894-64c9c222f8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:30.383130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 800/1644] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:30.383130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d69453-712b-4e27-9894-64c9c222f8ea None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:30.407696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d69453-712b-4e27-9894-64c9c222f8ea None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:30.418816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375baef-308c-45c6-b85f-59df115e4931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:30.427623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375baef-308c-45c6-b85f-59df115e4931 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:30.427623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375baef-308c-45c6-b85f-59df115e4931 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:30.427623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375baef-308c-45c6-b85f-59df115e4931 admin admin] PATH_INFO: `/v3/projects/b575d8ed68ab4b5eb9bec3bfd204991b/users/86cde6620728449f92adacc51f67df10/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:30.427623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375baef-308c-45c6-b85f-59df115e4931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b575d8ed68ab4b5eb9bec3bfd204991b, user_id=86cde6620728449f92adacc51f67df10, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:30.432217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375baef-308c-45c6-b85f-59df115e4931 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:30.454421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 801/1645] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:30 2022] PUT /identity/v3/projects/b575d8ed68ab4b5eb9bec3bfd204991b/users/86cde6620728449f92adacc51f67df10/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:30.456070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375baef-308c-45c6-b85f-59df115e4931 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:30.481604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0a31a-3c10-4b61-b5f6-a1b5ad56437f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:30.607676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c1055-af5e-4324-8cb1-0974e07ef5e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:30.609220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1055-af5e-4324-8cb1-0974e07ef5e0 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:30.609996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1055-af5e-4324-8cb1-0974e07ef5e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:30.610491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1055-af5e-4324-8cb1-0974e07ef5e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:30.660696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:30.680165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375baef-308c-45c6-b85f-59df115e4931 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:30.711729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f704a8e-befb-4bb5-9d04-cb02a6b80bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:30.712664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f704a8e-befb-4bb5-9d04-cb02a6b80bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:30.712949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f704a8e-befb-4bb5-9d04-cb02a6b80bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:30.713228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f704a8e-befb-4bb5-9d04-cb02a6b80bc8 admin admin] PATH_INFO: `/v3/projects/586cb24019004587b71d7817f45e7e56/users/affac764cb964a06ae30275fabc0edfd/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:30.720676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f704a8e-befb-4bb5-9d04-cb02a6b80bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=586cb24019004587b71d7817f45e7e56, user_id=affac764cb964a06ae30275fabc0edfd, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:30.725728 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f704a8e-befb-4bb5-9d04-cb02a6b80bc8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:30.739758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 802/1646] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:30 2022] PUT /identity/v3/projects/586cb24019004587b71d7817f45e7e56/users/affac764cb964a06ae30275fabc0edfd/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:30.741503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f704a8e-befb-4bb5-9d04-cb02a6b80bc8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:30.845201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c1055-af5e-4324-8cb1-0974e07ef5e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:30.851097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c1055-af5e-4324-8cb1-0974e07ef5e0 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:30.866740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 845/1647] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:30.871108 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c1055-af5e-4324-8cb1-0974e07ef5e0 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:30.878388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-626c1055-af5e-4324-8cb1-0974e07ef5e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:30.902989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f704a8e-befb-4bb5-9d04-cb02a6b80bc8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:30.921631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e52cd-b5c0-4665-86de-a0585cd14686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:30.922842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e52cd-b5c0-4665-86de-a0585cd14686 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:30.923139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e52cd-b5c0-4665-86de-a0585cd14686 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:30.923427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e52cd-b5c0-4665-86de-a0585cd14686 admin admin] PATH_INFO: `/v3/projects/1219f4d297154dbdb51c46f0a439d15d/users/87a2a0f44caf48469d7ca3d6fb78c86d/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:30.927888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e52cd-b5c0-4665-86de-a0585cd14686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1219f4d297154dbdb51c46f0a439d15d, user_id=87a2a0f44caf48469d7ca3d6fb78c86d, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:30.940494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e52cd-b5c0-4665-86de-a0585cd14686 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:30.981283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 803/1648] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:30 2022] PUT /identity/v3/projects/1219f4d297154dbdb51c46f0a439d15d/users/87a2a0f44caf48469d7ca3d6fb78c86d/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:30.985086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bbe2-406d-4f57-b6da-d05a974db7ee None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:30.986061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bbe2-406d-4f57-b6da-d05a974db7ee None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:30.986866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bbe2-406d-4f57-b6da-d05a974db7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:31.032362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c1055-af5e-4324-8cb1-0974e07ef5e0 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:31.059832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-3d10bbe2-406d-4f57-b6da-d05a974db7ee None None] MFA Rules not processed for user `86cde6620728449f92adacc51f67df10`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:31.062200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3d10bbe2-406d-4f57-b6da-d05a974db7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:31.196546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd374040-b131-4c97-8133-f7289a332ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:31.197627 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd374040-b131-4c97-8133-f7289a332ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:31.197923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd374040-b131-4c97-8133-f7289a332ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:31.198285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd374040-b131-4c97-8133-f7289a332ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:31.241713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 804/1649] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:31.246623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbafd7-50d8-426c-a0eb-2f3e566d378d None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:31.247162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbafd7-50d8-426c-a0eb-2f3e566d378d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:31.247511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbafd7-50d8-426c-a0eb-2f3e566d378d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:31.329768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-79fbafd7-50d8-426c-a0eb-2f3e566d378d None None] MFA Rules not processed for user `affac764cb964a06ae30275fabc0edfd`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:31.331397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-79fbafd7-50d8-426c-a0eb-2f3e566d378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:31.391555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd374040-b131-4c97-8133-f7289a332ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:31.400727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd374040-b131-4c97-8133-f7289a332ec1 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:31.429543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 846/1650] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:31.436764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0436a3-8ddb-4193-a3da-f4e7ca4bbd94 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:31.437524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0436a3-8ddb-4193-a3da-f4e7ca4bbd94 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:31.438211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0436a3-8ddb-4193-a3da-f4e7ca4bbd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:31.510588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 805/1651] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:31 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:31.511590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbafd7-50d8-426c-a0eb-2f3e566d378d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:31.513622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-79fbafd7-50d8-426c-a0eb-2f3e566d378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:31.545305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-fd0436a3-8ddb-4193-a3da-f4e7ca4bbd94 None None] MFA Rules not processed for user `87a2a0f44caf48469d7ca3d6fb78c86d`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:31.550410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fd0436a3-8ddb-4193-a3da-f4e7ca4bbd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:31.617125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbafd7-50d8-426c-a0eb-2f3e566d378d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:31.627153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a06db6-7603-4a9b-96fc-c93b01b93a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:31.627926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a06db6-7603-4a9b-96fc-c93b01b93a69 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:31.628584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a06db6-7603-4a9b-96fc-c93b01b93a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:31.629176 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a06db6-7603-4a9b-96fc-c93b01b93a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:31.645248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a06db6-7603-4a9b-96fc-c93b01b93a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:31.649371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a06db6-7603-4a9b-96fc-c93b01b93a69 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:31.676835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 806/1652] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:31.744301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a06db6-7603-4a9b-96fc-c93b01b93a69 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:31.760838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a06db6-7603-4a9b-96fc-c93b01b93a69 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:31.765184 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3a6e4-4e76-485c-995c-dd57f667c0b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:31.766014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3a6e4-4e76-485c-995c-dd57f667c0b1 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:31.766390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3a6e4-4e76-485c-995c-dd57f667c0b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:31.766773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3a6e4-4e76-485c-995c-dd57f667c0b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:31.795220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 847/1653] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:31.810810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3a6e4-4e76-485c-995c-dd57f667c0b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:31.814265 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3a6e4-4e76-485c-995c-dd57f667c0b1 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:31.837670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 807/1654] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:31 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:32.029411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0436a3-8ddb-4193-a3da-f4e7ca4bbd94 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:32.052439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0436a3-8ddb-4193-a3da-f4e7ca4bbd94 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:32.060414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20189f-2fd5-4b21-9a99-51163949c070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:32.062684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20189f-2fd5-4b21-9a99-51163949c070 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:32.063818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20189f-2fd5-4b21-9a99-51163949c070 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:32.064801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20189f-2fd5-4b21-9a99-51163949c070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:32.089561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20189f-2fd5-4b21-9a99-51163949c070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:32.093517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20189f-2fd5-4b21-9a99-51163949c070 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:32.133123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 848/1655] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:32.228317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:32.420836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:32.880150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3a6e4-4e76-485c-995c-dd57f667c0b1 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:32.913476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3a6e4-4e76-485c-995c-dd57f667c0b1 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:32.935473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:32.936267 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:32.943143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:32.943430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] PATH_INFO: `/v3/users/4fa4c7991f3247dc91b03cd61d303c09` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:32.952223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fa4c7991f3247dc91b03cd61d303c09)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:32.956053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:33.047272 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05337f9-5acb-4e69-9e54-55b70707ebc3 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:33.048021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05337f9-5acb-4e69-9e54-55b70707ebc3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:33.048584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05337f9-5acb-4e69-9e54-55b70707ebc3 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:33.051302 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 849/1656] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 4 12:02:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:02:33.052979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05337f9-5acb-4e69-9e54-55b70707ebc3 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:33.080546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05337f9-5acb-4e69-9e54-55b70707ebc3 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:33.107212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:33.109068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:33.109940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:33.110550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] PATH_INFO: `/v3/users/f1e591a6065a41a78a74591fcc9ab8e3` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:33.115877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e591a6065a41a78a74591fcc9ab8e3)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:33.124259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:33.276890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:33.400233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fa4c7991f3247dc91b03cd61d303c09'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:33.438833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e591a6065a41a78a74591fcc9ab8e3'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:33.439885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fa4c7991f3247dc91b03cd61d303c09'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:33.440635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e591a6065a41a78a74591fcc9ab8e3'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:33.442775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fa4c7991f3247dc91b03cd61d303c09'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:33.464415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e591a6065a41a78a74591fcc9ab8e3'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:33.465589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e591a6065a41a78a74591fcc9ab8e3'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:33.549627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fa4c7991f3247dc91b03cd61d303c09'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:33.581386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-5e6a4a2b-671a-4846-8d32-6ba886453b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fa4c7991f3247dc91b03cd61d303c09'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:33.599841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 808/1657] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:32 2022] DELETE /identity/v3/users/4fa4c7991f3247dc91b03cd61d303c09 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:33.599841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4316b-0f56-4be6-a95d-665058a23739 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:33.599841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4316b-0f56-4be6-a95d-665058a23739 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:33.599841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4316b-0f56-4be6-a95d-665058a23739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:33.649539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-0dc4316b-0f56-4be6-a95d-665058a23739 None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:33.650186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0dc4316b-0f56-4be6-a95d-665058a23739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:33.680905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-bdaf4f9a-7081-4252-b031-d124a1cdf4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e591a6065a41a78a74591fcc9ab8e3'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:33.760206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 850/1658] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:33 2022] DELETE /identity/v3/users/f1e591a6065a41a78a74591fcc9ab8e3 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:33.762422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92f094-9e18-422f-bf04-7a87ece9ae13 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:33.772680 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92f094-9e18-422f-bf04-7a87ece9ae13 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:33.773237 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92f094-9e18-422f-bf04-7a87ece9ae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:33.815531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 809/1659] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 4 12:02:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:33.817735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8476e1-fdc7-4799-8b95-83b2d3c03c69 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:33.818024 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8476e1-fdc7-4799-8b95-83b2d3c03c69 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:33.819503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8476e1-fdc7-4799-8b95-83b2d3c03c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:33.839095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-6f92f094-9e18-422f-bf04-7a87ece9ae13 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:33.841361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6f92f094-9e18-422f-bf04-7a87ece9ae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:33.842571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-8c8476e1-fdc7-4799-8b95-83b2d3c03c69 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:33.843918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8c8476e1-fdc7-4799-8b95-83b2d3c03c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:33.984062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 810/1660] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:33.987524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a449b-4c32-4270-b4a5-f8d833124e67 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:33.993078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a449b-4c32-4270-b4a5-f8d833124e67 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:33.993815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a449b-4c32-4270-b4a5-f8d833124e67 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:34.000620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 811/1661] 10.222.0.15 () {54 vars in 912 bytes} [Tue Oct 4 12:02:33 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:02:34.006089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17cce2-8bbd-4fb2-ad83-7892ca6588f9 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:34.006514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17cce2-8bbd-4fb2-ad83-7892ca6588f9 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:34.006815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17cce2-8bbd-4fb2-ad83-7892ca6588f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:34.037044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 851/1662] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:34.037044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92f094-9e18-422f-bf04-7a87ece9ae13 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:34.042269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6f92f094-9e18-422f-bf04-7a87ece9ae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:34.048742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-3f17cce2-8bbd-4fb2-ad83-7892ca6588f9 None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:34.050660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3f17cce2-8bbd-4fb2-ad83-7892ca6588f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:34.285173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92f094-9e18-422f-bf04-7a87ece9ae13 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:34.307684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8294a85-976a-44fa-b6ac-174fdeacb716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:34.308756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8294a85-976a-44fa-b6ac-174fdeacb716 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:34.309200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8294a85-976a-44fa-b6ac-174fdeacb716 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:34.309711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8294a85-976a-44fa-b6ac-174fdeacb716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:34.332354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 812/1663] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:02:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:02:34.335222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17cce2-8bbd-4fb2-ad83-7892ca6588f9 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:34.338100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3f17cce2-8bbd-4fb2-ad83-7892ca6588f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:34.368088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8294a85-976a-44fa-b6ac-174fdeacb716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:34.369716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8294a85-976a-44fa-b6ac-174fdeacb716 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:34.387189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17cce2-8bbd-4fb2-ad83-7892ca6588f9 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:34.396716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 852/1664] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:34.399861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8294a85-976a-44fa-b6ac-174fdeacb716 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:34.401532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:34.402818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:34.403261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:34.403940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a8294a85-976a-44fa-b6ac-174fdeacb716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:34.404062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] PATH_INFO: `/v3/projects/bd2c6c3369b14759bae51f7cff4549b0` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:34.406836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd2c6c3369b14759bae51f7cff4549b0)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:34.413445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:34.436868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8294a85-976a-44fa-b6ac-174fdeacb716 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:34.448346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640131d-62ce-4096-b2fe-f01365c5820f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:34.449258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640131d-62ce-4096-b2fe-f01365c5820f service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:34.449613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640131d-62ce-4096-b2fe-f01365c5820f service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:34.449958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640131d-62ce-4096-b2fe-f01365c5820f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:34.465133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640131d-62ce-4096-b2fe-f01365c5820f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:34.470824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640131d-62ce-4096-b2fe-f01365c5820f service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:34.501116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 853/1665] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:34.505059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640131d-62ce-4096-b2fe-f01365c5820f service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:34.511046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9640131d-62ce-4096-b2fe-f01365c5820f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:34.601097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd2c6c3369b14759bae51f7cff4549b0', 'request_id': 'req-a523ca51-ee4b-48c0-a1f2-423f144417ea'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:34.609588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd2c6c3369b14759bae51f7cff4549b0', 'request_id': 'req-a523ca51-ee4b-48c0-a1f2-423f144417ea'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:34.618287 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd2c6c3369b14759bae51f7cff4549b0', 'request_id': 'req-a523ca51-ee4b-48c0-a1f2-423f144417ea'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:34.651580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640131d-62ce-4096-b2fe-f01365c5820f service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:34.719916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:34.767722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697e73f-5cdd-409d-bb1d-bc7458c4ae49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:34.768363 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697e73f-5cdd-409d-bb1d-bc7458c4ae49 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:34.768714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697e73f-5cdd-409d-bb1d-bc7458c4ae49 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:34.769067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697e73f-5cdd-409d-bb1d-bc7458c4ae49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:34.899637 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697e73f-5cdd-409d-bb1d-bc7458c4ae49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:34.900791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697e73f-5cdd-409d-bb1d-bc7458c4ae49 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:34.916159 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 854/1666] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:34.920000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697e73f-5cdd-409d-bb1d-bc7458c4ae49 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:34.936612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3697e73f-5cdd-409d-bb1d-bc7458c4ae49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:34.954126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] The token cache is being invalidate because project bd2c6c3369b14759bae51f7cff4549b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:34.954464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:34.958815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:34.964020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 813/1667] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:34 2022] DELETE /identity/v3/projects/bd2c6c3369b14759bae51f7cff4549b0 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:34.967959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:34.969468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:35.168103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523ca51-ee4b-48c0-a1f2-423f144417ea admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:35.193278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:35.196884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:35.198450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:35.199219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] PATH_INFO: `/v3/users/7dabf36d8e36481da167d8e29ebbe7a5` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:35.201786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dabf36d8e36481da167d8e29ebbe7a5)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:35.202164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697e73f-5cdd-409d-bb1d-bc7458c4ae49 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:35.210923 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:35.223045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:35.224164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:35.224697 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:35.225267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] PATH_INFO: `/v3/users/513b7df0970e436ca894a31ae4321b5e` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:35.228977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=513b7df0970e436ca894a31ae4321b5e)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:35.239277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:35.569500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:35.575010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dabf36d8e36481da167d8e29ebbe7a5'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:35.615319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dabf36d8e36481da167d8e29ebbe7a5'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:35.616542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dabf36d8e36481da167d8e29ebbe7a5'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:35.766324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513b7df0970e436ca894a31ae4321b5e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:35.781315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513b7df0970e436ca894a31ae4321b5e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:35.822450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dabf36d8e36481da167d8e29ebbe7a5'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:35.836676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513b7df0970e436ca894a31ae4321b5e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:35.842354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513b7df0970e436ca894a31ae4321b5e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:35.855529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75f32c34-30e0-4ca3-a087-f8958f397c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dabf36d8e36481da167d8e29ebbe7a5'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:35.885931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 814/1668] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:34 2022] DELETE /identity/v3/users/7dabf36d8e36481da167d8e29ebbe7a5 => generated 0 bytes in 918 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:02:35.965775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513b7df0970e436ca894a31ae4321b5e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:36.022089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 855/1669] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:34 2022] DELETE /identity/v3/users/513b7df0970e436ca894a31ae4321b5e => generated 0 bytes in 1105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:36.028120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77bd6d-bbed-4b35-8688-c36bcf0f4c02 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:36.028430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77bd6d-bbed-4b35-8688-c36bcf0f4c02 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:36.028678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77bd6d-bbed-4b35-8688-c36bcf0f4c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:36.075466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-2b77bd6d-bbed-4b35-8688-c36bcf0f4c02 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:36.077892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2b77bd6d-bbed-4b35-8688-c36bcf0f4c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:36.264664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 815/1670] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:36.312738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:36.314509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:36.449677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a69bbe-7ca6-4646-adf2-dd61617f19d0 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:36.454262 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406ef9-e778-41e4-8aac-b839312bf103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:36.455230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406ef9-e778-41e4-8aac-b839312bf103 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:36.455658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406ef9-e778-41e4-8aac-b839312bf103 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:36.456111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406ef9-e778-41e4-8aac-b839312bf103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:36.479709 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa406ef9-e778-41e4-8aac-b839312bf103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:36.481439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa406ef9-e778-41e4-8aac-b839312bf103 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:36.503891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 856/1671] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:36.555119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77bd6d-bbed-4b35-8688-c36bcf0f4c02 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:36.558917 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2b77bd6d-bbed-4b35-8688-c36bcf0f4c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:36.640209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77bd6d-bbed-4b35-8688-c36bcf0f4c02 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:36.653332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:36.654239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:36.654672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:36.655065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] PATH_INFO: `/v3/projects/05221a6000364bfea83e82d2eba93b1a` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:36.669438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05221a6000364bfea83e82d2eba93b1a)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:36.672444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:36.877442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05221a6000364bfea83e82d2eba93b1a', 'request_id': 'req-8ce07444-e2bf-4249-8b4e-57c2f934b486'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:36.921837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05221a6000364bfea83e82d2eba93b1a', 'request_id': 'req-8ce07444-e2bf-4249-8b4e-57c2f934b486'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:36.939548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05221a6000364bfea83e82d2eba93b1a', 'request_id': 'req-8ce07444-e2bf-4249-8b4e-57c2f934b486'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:37.107749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406ef9-e778-41e4-8aac-b839312bf103 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:37.109549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] The token cache is being invalidate because project 05221a6000364bfea83e82d2eba93b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:37.109854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:37.110501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fa406ef9-e778-41e4-8aac-b839312bf103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:37.111932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:37.115278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 816/1672] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:36 2022] DELETE /identity/v3/projects/05221a6000364bfea83e82d2eba93b1a => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:37.122363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:37.124829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:37.162582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406ef9-e778-41e4-8aac-b839312bf103 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:37.356480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce07444-e2bf-4249-8b4e-57c2f934b486 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:37.369070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:37.370193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:37.370688 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:37.371197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] PATH_INFO: `/v3/users/86cde6620728449f92adacc51f67df10` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:37.379340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86cde6620728449f92adacc51f67df10)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:37.383365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:37.442063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:37.443153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:37.443682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:37.444149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] PATH_INFO: `/v3/projects/052474df3ff8460ea78de214416439ec` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:37.447587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:37.454243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=052474df3ff8460ea78de214416439ec)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:37.456358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: WARNING py.warnings [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:02:37.456358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: warnings.warn(msg) Oct 04 12:02:37.456358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: Oct 04 12:02:37.459910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:37.527009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:37.830960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86cde6620728449f92adacc51f67df10'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:37.841828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '052474df3ff8460ea78de214416439ec', 'request_id': 'req-44b7f49c-f591-4766-8498-0858cea6fbd0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:37.858283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86cde6620728449f92adacc51f67df10'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:37.861115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86cde6620728449f92adacc51f67df10'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:37.862741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '052474df3ff8460ea78de214416439ec', 'request_id': 'req-44b7f49c-f591-4766-8498-0858cea6fbd0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:37.959903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '052474df3ff8460ea78de214416439ec', 'request_id': 'req-44b7f49c-f591-4766-8498-0858cea6fbd0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:38.006721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] The token cache is being invalidate because project 052474df3ff8460ea78de214416439ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:38.007341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:38.008190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:38.011652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 857/1673] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:37 2022] DELETE /identity/v3/projects/052474df3ff8460ea78de214416439ec => generated 0 bytes in 905 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:38.015151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:38.021205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:38.051533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86cde6620728449f92adacc51f67df10'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:38.091660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86cde6620728449f92adacc51f67df10'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:38.145616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 817/1674] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:37 2022] DELETE /identity/v3/users/86cde6620728449f92adacc51f67df10 => generated 0 bytes in 1018 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:02:38.152136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:38.155317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:38.218985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:38.235615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7f49c-f591-4766-8498-0858cea6fbd0 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:38.420666 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db3b93-1563-41b5-b0e3-dc63c96bf798 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:38.432827 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:38.441039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:38.441039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:38.441039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] PATH_INFO: `/v3/users/affac764cb964a06ae30275fabc0edfd` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:38.441039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=affac764cb964a06ae30275fabc0edfd)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:38.454975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:38.534173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:38.535843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:38.536588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:38.537865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] PATH_INFO: `/v3/users/210399c956734c76bb78137afd48dafd` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:38.540689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=210399c956734c76bb78137afd48dafd)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:38.545088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:38.977078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '210399c956734c76bb78137afd48dafd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:38.980351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '210399c956734c76bb78137afd48dafd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:39.012043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'affac764cb964a06ae30275fabc0edfd'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:39.025739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '210399c956734c76bb78137afd48dafd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:39.027732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '210399c956734c76bb78137afd48dafd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:39.056103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'affac764cb964a06ae30275fabc0edfd'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:39.058588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'affac764cb964a06ae30275fabc0edfd'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:39.228704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-c01f1f06-c4e2-43b8-8d6b-3c3b31b8e4c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '210399c956734c76bb78137afd48dafd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:39.273374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 858/1675] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:38 2022] DELETE /identity/v3/users/210399c956734c76bb78137afd48dafd => generated 0 bytes in 1262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:39.281054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de69e5-e132-4bb1-8144-4c27165b5b85 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:39.281631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de69e5-e132-4bb1-8144-4c27165b5b85 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:39.282301 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de69e5-e132-4bb1-8144-4c27165b5b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:39.354532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'affac764cb964a06ae30275fabc0edfd'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:39.364290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b4de69e5-e132-4bb1-8144-4c27165b5b85 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:39.370910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b4de69e5-e132-4bb1-8144-4c27165b5b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:39.450377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'affac764cb964a06ae30275fabc0edfd'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:39.502568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 818/1676] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:38 2022] DELETE /identity/v3/users/affac764cb964a06ae30275fabc0edfd => generated 0 bytes in 1351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:39.504784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:39.509352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:39.639347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 859/1677] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:39.643615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de69e5-e132-4bb1-8144-4c27165b5b85 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:39.647913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b4de69e5-e132-4bb1-8144-4c27165b5b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:39.710768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de69e5-e132-4bb1-8144-4c27165b5b85 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:39.864829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0d745-12c2-4902-b665-e313cec843a8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:39.907475 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:39.908728 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:39.909507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:39.910751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] PATH_INFO: `/v3/projects/57c344f95027402e8598ed54ad453c7f` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:39.913277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c344f95027402e8598ed54ad453c7f)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:39.921633 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:40.018361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:40.045936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:40.047063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:40.047555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:40.048322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] PATH_INFO: `/v3/projects/9741380a634142c1b9b4065cdf7a15f9` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:40.051339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9741380a634142c1b9b4065cdf7a15f9)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:40.055188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:40.256982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c344f95027402e8598ed54ad453c7f', 'request_id': 'req-e1508316-445d-4038-aff3-d18b8b31fa28'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:40.297681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c344f95027402e8598ed54ad453c7f', 'request_id': 'req-e1508316-445d-4038-aff3-d18b8b31fa28'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:40.330254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9741380a634142c1b9b4065cdf7a15f9', 'request_id': 'req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:40.341389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c344f95027402e8598ed54ad453c7f', 'request_id': 'req-e1508316-445d-4038-aff3-d18b8b31fa28'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:40.347536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9741380a634142c1b9b4065cdf7a15f9', 'request_id': 'req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:40.446555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9741380a634142c1b9b4065cdf7a15f9', 'request_id': 'req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:40.467737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] The token cache is being invalidate because project 9741380a634142c1b9b4065cdf7a15f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:40.469228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:40.470531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:40.477882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 860/1678] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:39 2022] DELETE /identity/v3/projects/9741380a634142c1b9b4065cdf7a15f9 => generated 0 bytes in 833 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:40.479233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:40.481372 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:40.584621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] The token cache is being invalidate because project 57c344f95027402e8598ed54ad453c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:40.587624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:40.591648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:40.603629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 819/1679] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:39 2022] DELETE /identity/v3/projects/57c344f95027402e8598ed54ad453c7f => generated 0 bytes in 1096 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:40.605525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:40.610484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:40.654549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fc8e7-8d39-4981-a6ff-5adc5892f50e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:40.668423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e682b5-8706-4a9a-80cd-b12b23effe21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:40.670405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e682b5-8706-4a9a-80cd-b12b23effe21 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:40.671380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e682b5-8706-4a9a-80cd-b12b23effe21 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:40.672139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e682b5-8706-4a9a-80cd-b12b23effe21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:40.676969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c2e682b5-8706-4a9a-80cd-b12b23effe21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:40.833114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1508316-445d-4038-aff3-d18b8b31fa28 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:40.858456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:40.995220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:40.996078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:40.996335 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:40.996481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] PATH_INFO: `/v3/projects/b575d8ed68ab4b5eb9bec3bfd204991b` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:40.999631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b575d8ed68ab4b5eb9bec3bfd204991b)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:41.002880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:41.045168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e682b5-8706-4a9a-80cd-b12b23effe21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:41.047383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e682b5-8706-4a9a-80cd-b12b23effe21 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:41.092563 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 861/1680] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:41.094938 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc7b84-44ea-457c-8372-2f9f90d56bb7 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:41.095288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc7b84-44ea-457c-8372-2f9f90d56bb7 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:41.095564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc7b84-44ea-457c-8372-2f9f90d56bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:41.146709 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f6cc7b84-44ea-457c-8372-2f9f90d56bb7 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:41.148031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f6cc7b84-44ea-457c-8372-2f9f90d56bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:41.187368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 862/1681] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:41.187368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc7b84-44ea-457c-8372-2f9f90d56bb7 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:41.189507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f6cc7b84-44ea-457c-8372-2f9f90d56bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:41.223698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc7b84-44ea-457c-8372-2f9f90d56bb7 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:41.250023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b575d8ed68ab4b5eb9bec3bfd204991b', 'request_id': 'req-8327d975-4934-4c34-8708-64b84f3468e8'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:41.261191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b575d8ed68ab4b5eb9bec3bfd204991b', 'request_id': 'req-8327d975-4934-4c34-8708-64b84f3468e8'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:41.295253 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b575d8ed68ab4b5eb9bec3bfd204991b', 'request_id': 'req-8327d975-4934-4c34-8708-64b84f3468e8'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:41.497359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:41.498277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:41.498723 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:41.499143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] PATH_INFO: `/v3/users/87a2a0f44caf48469d7ca3d6fb78c86d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:41.500952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a2a0f44caf48469d7ca3d6fb78c86d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:41.504356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:41.682170 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] The token cache is being invalidate because project b575d8ed68ab4b5eb9bec3bfd204991b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:41.682508 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:41.683516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8327d975-4934-4c34-8708-64b84f3468e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:41.688357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 820/1682] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:40 2022] DELETE /identity/v3/projects/b575d8ed68ab4b5eb9bec3bfd204991b => generated 0 bytes in 1084 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:41.690496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061f3d9-2d4f-4c0d-9ab8-0ac6560c5130 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:41.696619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061f3d9-2d4f-4c0d-9ab8-0ac6560c5130 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:41.696619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061f3d9-2d4f-4c0d-9ab8-0ac6560c5130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:41.739690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f061f3d9-2d4f-4c0d-9ab8-0ac6560c5130 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:41.741097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f061f3d9-2d4f-4c0d-9ab8-0ac6560c5130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:41.766462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 821/1683] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:41.768015 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faafedd-5c9f-4c6a-880e-c047e65cccfa None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:41.769832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faafedd-5c9f-4c6a-880e-c047e65cccfa None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:41.771554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faafedd-5c9f-4c6a-880e-c047e65cccfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:41.837698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-5faafedd-5c9f-4c6a-880e-c047e65cccfa None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:41.837698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5faafedd-5c9f-4c6a-880e-c047e65cccfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:41.845068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 822/1684] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:41.845932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faafedd-5c9f-4c6a-880e-c047e65cccfa None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:41.869701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faafedd-5c9f-4c6a-880e-c047e65cccfa None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:41.876006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0a2e8-77ba-4253-9b00-6e17fe5fa5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:41.877119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0a2e8-77ba-4253-9b00-6e17fe5fa5db admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:41.877565 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0a2e8-77ba-4253-9b00-6e17fe5fa5db admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:41.877948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0a2e8-77ba-4253-9b00-6e17fe5fa5db admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:41.879206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0a2e8-77ba-4253-9b00-6e17fe5fa5db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:41.879661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0a2e8-77ba-4253-9b00-6e17fe5fa5db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:41.883045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0a2e8-77ba-4253-9b00-6e17fe5fa5db admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:41.913639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 823/1685] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:41.923632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aca10f-2917-43af-90ec-c446ad971937 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:41.924018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aca10f-2917-43af-90ec-c446ad971937 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:41.924218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aca10f-2917-43af-90ec-c446ad971937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:42.015027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:42.050535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-86aca10f-2917-43af-90ec-c446ad971937 None None] MFA Rules not processed for user `fd040e13fd6c4b0dbea6a65932f84cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:42.065949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-86aca10f-2917-43af-90ec-c446ad971937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:42.088094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a2a0f44caf48469d7ca3d6fb78c86d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:42.091639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a2a0f44caf48469d7ca3d6fb78c86d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:42.129968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a2a0f44caf48469d7ca3d6fb78c86d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:42.131330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a2a0f44caf48469d7ca3d6fb78c86d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:42.264601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a2a0f44caf48469d7ca3d6fb78c86d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:42.409434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 824/1687] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:42.409971 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 863/1687] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:41 2022] DELETE /identity/v3/users/87a2a0f44caf48469d7ca3d6fb78c86d => generated 0 bytes in 1222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:42.415859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:42.416041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccef52-fae6-448d-9205-db02bb0e3876 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:42.416119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccef52-fae6-448d-9205-db02bb0e3876 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:42.420958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:42.422267 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccef52-fae6-448d-9205-db02bb0e3876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:42.497803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-5cccef52-fae6-448d-9205-db02bb0e3876 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:42.522575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5cccef52-fae6-448d-9205-db02bb0e3876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:42.629383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409869d2-89c4-4406-adc5-922a24cafd1a admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:42.643005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0985e2-9cb4-4bb3-a23d-82fe79d9aad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:42.643661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0985e2-9cb4-4bb3-a23d-82fe79d9aad9 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:42.647962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0985e2-9cb4-4bb3-a23d-82fe79d9aad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:42.648614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0985e2-9cb4-4bb3-a23d-82fe79d9aad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:42.649783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0985e2-9cb4-4bb3-a23d-82fe79d9aad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:42.657867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0985e2-9cb4-4bb3-a23d-82fe79d9aad9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:42.780564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 825/1688] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:42.790753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccef52-fae6-448d-9205-db02bb0e3876 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:42.793231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5cccef52-fae6-448d-9205-db02bb0e3876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:42.835379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 864/1689] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:42 2022] POST /identity/v3/projects => generated 385 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:42.842700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0985e2-9cb4-4bb3-a23d-82fe79d9aad9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:42.854632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3e0985e2-9cb4-4bb3-a23d-82fe79d9aad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:42.934706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccef52-fae6-448d-9205-db02bb0e3876 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:43.040227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:43.098851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0985e2-9cb4-4bb3-a23d-82fe79d9aad9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:43.108415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560fac8-7c06-48e5-a29d-a723e00cb4e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:43.109403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560fac8-7c06-48e5-a29d-a723e00cb4e1 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:43.109709 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560fac8-7c06-48e5-a29d-a723e00cb4e1 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:43.110042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560fac8-7c06-48e5-a29d-a723e00cb4e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:43.114379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:43.116483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:43.116799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:43.117220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] PATH_INFO: `/v3/projects/586cb24019004587b71d7817f45e7e56` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:43.121698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=586cb24019004587b71d7817f45e7e56)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:43.124150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:43.210628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560fac8-7c06-48e5-a29d-a723e00cb4e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:43.212694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560fac8-7c06-48e5-a29d-a723e00cb4e1 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:43.238789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 865/1690] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:43.239714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560fac8-7c06-48e5-a29d-a723e00cb4e1 service nova] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:43.243895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9560fac8-7c06-48e5-a29d-a723e00cb4e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:43.276024 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560fac8-7c06-48e5-a29d-a723e00cb4e1 service nova] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:43.285648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:43.291299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:43.291299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:43.291299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] PATH_INFO: `/v3/projects/20f2e63652624b738fbc342edeedeb62` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:43.291299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20f2e63652624b738fbc342edeedeb62)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:43.292026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:43.354375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '586cb24019004587b71d7817f45e7e56', 'request_id': 'req-800eda5e-09bf-4fbc-b024-193f64208f16'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:43.370880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '586cb24019004587b71d7817f45e7e56', 'request_id': 'req-800eda5e-09bf-4fbc-b024-193f64208f16'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:43.386377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '586cb24019004587b71d7817f45e7e56', 'request_id': 'req-800eda5e-09bf-4fbc-b024-193f64208f16'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:43.532783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] The token cache is being invalidate because project 586cb24019004587b71d7817f45e7e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:43.533956 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:43.541896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:43.544701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 826/1691] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:42 2022] DELETE /identity/v3/projects/586cb24019004587b71d7817f45e7e56 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:43.546068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:43.547938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:43.559663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20f2e63652624b738fbc342edeedeb62', 'request_id': 'req-aaee4b38-9708-4dde-bfa0-de70033c6cd5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:43.567873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20f2e63652624b738fbc342edeedeb62', 'request_id': 'req-aaee4b38-9708-4dde-bfa0-de70033c6cd5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:43.694867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800eda5e-09bf-4fbc-b024-193f64208f16 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:43.697060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20f2e63652624b738fbc342edeedeb62', 'request_id': 'req-aaee4b38-9708-4dde-bfa0-de70033c6cd5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:43.726702 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] The token cache is being invalidate because project 20f2e63652624b738fbc342edeedeb62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:43.727380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:43.729332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:43.731589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 866/1692] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:43 2022] DELETE /identity/v3/projects/20f2e63652624b738fbc342edeedeb62 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:43.743952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:43.747964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:43.871408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7c59a-702f-489d-afdb-79105d80a25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:43.872478 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c59a-702f-489d-afdb-79105d80a25e admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:43.872933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c59a-702f-489d-afdb-79105d80a25e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:43.873360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c59a-702f-489d-afdb-79105d80a25e admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:43.874439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7c59a-702f-489d-afdb-79105d80a25e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:43.875157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7c59a-702f-489d-afdb-79105d80a25e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:43.888933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7c59a-702f-489d-afdb-79105d80a25e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:43.905306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 827/1693] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:43.906623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb3fe4-0f50-4e28-8459-aedd6ad7a533 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:43.907078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb3fe4-0f50-4e28-8459-aedd6ad7a533 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:43.907362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb3fe4-0f50-4e28-8459-aedd6ad7a533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:43.973985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-55eb3fe4-0f50-4e28-8459-aedd6ad7a533 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:43.980359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-55eb3fe4-0f50-4e28-8459-aedd6ad7a533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:44.004277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 828/1694] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:44.007135 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c4b93-47f1-4569-8838-a676f3fdc81e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:44.007596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c4b93-47f1-4569-8838-a676f3fdc81e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:44.008066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c4b93-47f1-4569-8838-a676f3fdc81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:44.039915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee4b38-9708-4dde-bfa0-de70033c6cd5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:44.074512 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df59ef-19f7-4329-90c3-6911bb90aef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:44.075746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df59ef-19f7-4329-90c3-6911bb90aef0 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:44.076465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df59ef-19f7-4329-90c3-6911bb90aef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:44.077312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df59ef-19f7-4329-90c3-6911bb90aef0 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:44.078718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df59ef-19f7-4329-90c3-6911bb90aef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:44.084742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df59ef-19f7-4329-90c3-6911bb90aef0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:44.089894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-6c8c4b93-47f1-4569-8838-a676f3fdc81e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:44.093461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-6c8c4b93-47f1-4569-8838-a676f3fdc81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:44.112522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 829/1695] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:44.113890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c4b93-47f1-4569-8838-a676f3fdc81e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:44.120309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-6c8c4b93-47f1-4569-8838-a676f3fdc81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:44.179943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c4b93-47f1-4569-8838-a676f3fdc81e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:44.193587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:44.196873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:44.203980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:44.204826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] PATH_INFO: `/v3/projects/1219f4d297154dbdb51c46f0a439d15d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:44.207955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1219f4d297154dbdb51c46f0a439d15d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:44.214028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:44.319964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 867/1696] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:43 2022] POST /identity/v3/users => generated 293 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:44.321914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df59ef-19f7-4329-90c3-6911bb90aef0 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:44.325508 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b2df59ef-19f7-4329-90c3-6911bb90aef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:44.350800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df59ef-19f7-4329-90c3-6911bb90aef0 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:44.363055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3206b-3769-4f65-b480-1b54b1dc5020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:44.364629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3206b-3769-4f65-b480-1b54b1dc5020 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:44.365424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3206b-3769-4f65-b480-1b54b1dc5020 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:44.366037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3206b-3769-4f65-b480-1b54b1dc5020 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:44.367574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3206b-3769-4f65-b480-1b54b1dc5020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:44.371765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3206b-3769-4f65-b480-1b54b1dc5020 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:44.478495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1219f4d297154dbdb51c46f0a439d15d', 'request_id': 'req-001680a5-8520-4251-ab3f-d277e8232692'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:44.507949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1219f4d297154dbdb51c46f0a439d15d', 'request_id': 'req-001680a5-8520-4251-ab3f-d277e8232692'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:44.516679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1219f4d297154dbdb51c46f0a439d15d', 'request_id': 'req-001680a5-8520-4251-ab3f-d277e8232692'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:44.535964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 868/1697] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:44 2022] POST /identity/v3/projects => generated 377 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:44.539395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fc6f0-b7a2-4d1d-b5e9-17fb9054c263 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:44.540193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fc6f0-b7a2-4d1d-b5e9-17fb9054c263 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:44.541427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fc6f0-b7a2-4d1d-b5e9-17fb9054c263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:44.605401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-cb0fc6f0-b7a2-4d1d-b5e9-17fb9054c263 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:44.607929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cb0fc6f0-b7a2-4d1d-b5e9-17fb9054c263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:44.662759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] The token cache is being invalidate because project 1219f4d297154dbdb51c46f0a439d15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:44.662759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:44.662759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-001680a5-8520-4251-ab3f-d277e8232692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:44.662759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 830/1698] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:44 2022] DELETE /identity/v3/projects/1219f4d297154dbdb51c46f0a439d15d => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:44.662759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f0e29-1294-4fbd-b38d-c7b5e71f4ff6 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:44.664097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f0e29-1294-4fbd-b38d-c7b5e71f4ff6 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:44.664097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f0e29-1294-4fbd-b38d-c7b5e71f4ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:44.689194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7c7f0e29-1294-4fbd-b38d-c7b5e71f4ff6 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:44.690524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7c7f0e29-1294-4fbd-b38d-c7b5e71f4ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:44.738273 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 869/1699] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:44.741355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fc6f0-b7a2-4d1d-b5e9-17fb9054c263 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:44.743557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cb0fc6f0-b7a2-4d1d-b5e9-17fb9054c263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:44.762867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fc6f0-b7a2-4d1d-b5e9-17fb9054c263 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:44.769486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585e2ea-7805-4173-aab9-05b6791b06d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:44.770330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e2ea-7805-4173-aab9-05b6791b06d0 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:44.770785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e2ea-7805-4173-aab9-05b6791b06d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:44.772035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e2ea-7805-4173-aab9-05b6791b06d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:44.772939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585e2ea-7805-4173-aab9-05b6791b06d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:44.775489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585e2ea-7805-4173-aab9-05b6791b06d0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:44.777831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 831/1700] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:44.778894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f0e29-1294-4fbd-b38d-c7b5e71f4ff6 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:44.780492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7c7f0e29-1294-4fbd-b38d-c7b5e71f4ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:44.803860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 870/1701] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:44.806343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3e0c3-6b96-4bde-b6ff-1abf3c510a25 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:44.806516 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3e0c3-6b96-4bde-b6ff-1abf3c510a25 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:44.806876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3e0c3-6b96-4bde-b6ff-1abf3c510a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:44.820647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:44.830992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f0e29-1294-4fbd-b38d-c7b5e71f4ff6 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:44.843359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72085b74-898e-40eb-88bf-b580b43a7356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:44.844379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085b74-898e-40eb-88bf-b580b43a7356 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:44.844594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085b74-898e-40eb-88bf-b580b43a7356 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:44.844948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085b74-898e-40eb-88bf-b580b43a7356 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:44.845696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72085b74-898e-40eb-88bf-b580b43a7356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:44.846813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-5bd3e0c3-6b96-4bde-b6ff-1abf3c510a25 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:44.849169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5bd3e0c3-6b96-4bde-b6ff-1abf3c510a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:44.850320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72085b74-898e-40eb-88bf-b580b43a7356 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:44.872264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 871/1702] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:44.872264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7e952-56da-4876-b756-aaf79dbcc784 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:44.872264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7e952-56da-4876-b756-aaf79dbcc784 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:44.872264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7e952-56da-4876-b756-aaf79dbcc784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:44.950178 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d5e7e952-56da-4876-b756-aaf79dbcc784 None None] MFA Rules not processed for user `64e03bf24bb44add8adea800932e2f75`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:44.951466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d5e7e952-56da-4876-b756-aaf79dbcc784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:45.042615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 832/1703] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:44 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:45.043496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72085b74-898e-40eb-88bf-b580b43a7356 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:45.074990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72085b74-898e-40eb-88bf-b580b43a7356 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:45.092030 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae3c8c-86d6-4441-bbf8-0028a962b113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:45.094464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae3c8c-86d6-4441-bbf8-0028a962b113 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.094464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae3c8c-86d6-4441-bbf8-0028a962b113 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.094464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae3c8c-86d6-4441-bbf8-0028a962b113 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.094464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae3c8c-86d6-4441-bbf8-0028a962b113 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:45.094464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae3c8c-86d6-4441-bbf8-0028a962b113 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:45.095914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae3c8c-86d6-4441-bbf8-0028a962b113 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:45.127335 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 833/1704] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:45.128845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae3c8c-86d6-4441-bbf8-0028a962b113 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:45.151392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae3c8c-86d6-4441-bbf8-0028a962b113 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:45.166978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1733d-a9a4-4018-9eb1-2650ddac2200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:45.167565 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1733d-a9a4-4018-9eb1-2650ddac2200 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.167847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1733d-a9a4-4018-9eb1-2650ddac2200 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.168110 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1733d-a9a4-4018-9eb1-2650ddac2200 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.168826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1733d-a9a4-4018-9eb1-2650ddac2200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:45.169127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1733d-a9a4-4018-9eb1-2650ddac2200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:45.170745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1733d-a9a4-4018-9eb1-2650ddac2200 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:45.197805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 872/1705] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:45.202387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7e952-56da-4876-b756-aaf79dbcc784 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:45.203855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 834/1706] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:45.211741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f00aba-0e5f-4e2b-ba98-cdde9098c22a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.211975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f00aba-0e5f-4e2b-ba98-cdde9098c22a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.212364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f00aba-0e5f-4e2b-ba98-cdde9098c22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.247091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7e952-56da-4876-b756-aaf79dbcc784 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:45.258472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238f16d-40eb-4a41-ba52-7d5c305f6307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:45.261965 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238f16d-40eb-4a41-ba52-7d5c305f6307 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.262283 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238f16d-40eb-4a41-ba52-7d5c305f6307 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.262521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238f16d-40eb-4a41-ba52-7d5c305f6307 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.263199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238f16d-40eb-4a41-ba52-7d5c305f6307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:45.264888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238f16d-40eb-4a41-ba52-7d5c305f6307 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:45.294043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 873/1707] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:45.294963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238f16d-40eb-4a41-ba52-7d5c305f6307 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:45.301624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e5f00aba-0e5f-4e2b-ba98-cdde9098c22a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:45.303643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e5f00aba-0e5f-4e2b-ba98-cdde9098c22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:45.316703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238f16d-40eb-4a41-ba52-7d5c305f6307 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:45.326592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 835/1708] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:45.328001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f00aba-0e5f-4e2b-ba98-cdde9098c22a None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:45.333226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2f3df-5e04-45dc-94a4-9ce0be6b86b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:45.340820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2f3df-5e04-45dc-94a4-9ce0be6b86b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.340820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2f3df-5e04-45dc-94a4-9ce0be6b86b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.340820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2f3df-5e04-45dc-94a4-9ce0be6b86b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.340820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2f3df-5e04-45dc-94a4-9ce0be6b86b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:45.340820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2f3df-5e04-45dc-94a4-9ce0be6b86b2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:45.355764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 874/1709] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:45.357105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2f3df-5e04-45dc-94a4-9ce0be6b86b2 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:45.366950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f00aba-0e5f-4e2b-ba98-cdde9098c22a None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:45.371333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2f3df-5e04-45dc-94a4-9ce0be6b86b2 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:45.382529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa51f-72e2-4c9f-93f6-9c540ba70707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:45.382787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c245b5-4868-4c43-9320-b6bdbce66dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:45.383234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa51f-72e2-4c9f-93f6-9c540ba70707 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.384030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa51f-72e2-4c9f-93f6-9c540ba70707 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.384030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa51f-72e2-4c9f-93f6-9c540ba70707 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.384630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fa51f-72e2-4c9f-93f6-9c540ba70707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:45.385720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c245b5-4868-4c43-9320-b6bdbce66dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.386858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fa51f-72e2-4c9f-93f6-9c540ba70707 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:45.387657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c245b5-4868-4c43-9320-b6bdbce66dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.389965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c245b5-4868-4c43-9320-b6bdbce66dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.391413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c245b5-4868-4c43-9320-b6bdbce66dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:45.394343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c245b5-4868-4c43-9320-b6bdbce66dd4 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:45.529064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 875/1710] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:45 2022] POST /identity/v3/projects => generated 363 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:45.531619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905671f1-f442-44a4-99b2-f4bb2ab1865c None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.545018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905671f1-f442-44a4-99b2-f4bb2ab1865c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.545493 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905671f1-f442-44a4-99b2-f4bb2ab1865c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.588509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 836/1711] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:45 2022] POST /identity/v3/projects => generated 369 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:45.590488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c245b5-4868-4c43-9320-b6bdbce66dd4 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:45.600536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-905671f1-f442-44a4-99b2-f4bb2ab1865c None None] MFA Rules not processed for user `64e03bf24bb44add8adea800932e2f75`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:45.602579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-905671f1-f442-44a4-99b2-f4bb2ab1865c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:45.741251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 876/1712] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:45.742354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905671f1-f442-44a4-99b2-f4bb2ab1865c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:45.830575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c245b5-4868-4c43-9320-b6bdbce66dd4 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:45.842307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46015daf-2517-4925-ab0d-927b1a567718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:45.844476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46015daf-2517-4925-ab0d-927b1a567718 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.845260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46015daf-2517-4925-ab0d-927b1a567718 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.848805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46015daf-2517-4925-ab0d-927b1a567718 admin admin] PATH_INFO: `/v3/projects/855a8b5a9d45448486790dc490720037/users/a07f995e1b17487d91a3ab109a7cd926/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.881483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905671f1-f442-44a4-99b2-f4bb2ab1865c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:45.887953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb99c-6598-416e-947f-a254e46a809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:45.888818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb99c-6598-416e-947f-a254e46a809e admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.892238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb99c-6598-416e-947f-a254e46a809e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.892238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb99c-6598-416e-947f-a254e46a809e admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.892238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb99c-6598-416e-947f-a254e46a809e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:02:45.892238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb99c-6598-416e-947f-a254e46a809e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:45.893010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb99c-6598-416e-947f-a254e46a809e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:45.924972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46015daf-2517-4925-ab0d-927b1a567718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=855a8b5a9d45448486790dc490720037, user_id=a07f995e1b17487d91a3ab109a7cd926, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:45.931055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46015daf-2517-4925-ab0d-927b1a567718 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:45.933234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:45.934210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 877/1713] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:02:45.935646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb99c-6598-416e-947f-a254e46a809e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:45.967449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb99c-6598-416e-947f-a254e46a809e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:45.981862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1795efe-7949-4ba0-9e8b-ec35c86c5dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:45.981862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1795efe-7949-4ba0-9e8b-ec35c86c5dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:45.981862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1795efe-7949-4ba0-9e8b-ec35c86c5dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:45.982157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1795efe-7949-4ba0-9e8b-ec35c86c5dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:45.984356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1795efe-7949-4ba0-9e8b-ec35c86c5dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:45.989191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1795efe-7949-4ba0-9e8b-ec35c86c5dc5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:46.014590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 878/1714] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:46.014590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1795efe-7949-4ba0-9e8b-ec35c86c5dc5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:46.043214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1795efe-7949-4ba0-9e8b-ec35c86c5dc5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:46.060041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5671e-9cc4-438f-8b5c-58dd6e6be59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:46.060041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5671e-9cc4-438f-8b5c-58dd6e6be59c admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:46.060041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5671e-9cc4-438f-8b5c-58dd6e6be59c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:46.060041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5671e-9cc4-438f-8b5c-58dd6e6be59c admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:46.060663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5671e-9cc4-438f-8b5c-58dd6e6be59c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:46.062716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5671e-9cc4-438f-8b5c-58dd6e6be59c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:46.089038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 837/1715] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:45 2022] PUT /identity/v3/projects/855a8b5a9d45448486790dc490720037/users/a07f995e1b17487d91a3ab109a7cd926/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:46.090017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46015daf-2517-4925-ab0d-927b1a567718 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:46.232653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46015daf-2517-4925-ab0d-927b1a567718 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:46.243189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63e031-2168-421f-946c-2afd2956149f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:46.244019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63e031-2168-421f-946c-2afd2956149f admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:46.244458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63e031-2168-421f-946c-2afd2956149f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:46.244836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63e031-2168-421f-946c-2afd2956149f admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:46.245608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63e031-2168-421f-946c-2afd2956149f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:46.246717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 879/1716] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:46 2022] POST /identity/v3/users => generated 282 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:46.248993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63e031-2168-421f-946c-2afd2956149f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:46.249193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5671e-9cc4-438f-8b5c-58dd6e6be59c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:46.265694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c3e5671e-9cc4-438f-8b5c-58dd6e6be59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:46.369740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 838/1717] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:46 2022] POST /identity/v3/users => generated 285 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:46.369740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63e031-2168-421f-946c-2afd2956149f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:46.419317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63e031-2168-421f-946c-2afd2956149f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:46.434634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b087cb-f881-4e06-ae85-9e235d959d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:46.437953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b087cb-f881-4e06-ae85-9e235d959d9a admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:46.438308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b087cb-f881-4e06-ae85-9e235d959d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:46.438595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b087cb-f881-4e06-ae85-9e235d959d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:46.439324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b087cb-f881-4e06-ae85-9e235d959d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:46.441811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b087cb-f881-4e06-ae85-9e235d959d9a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:46.469413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5671e-9cc4-438f-8b5c-58dd6e6be59c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:46.580237 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 839/1718] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:46 2022] POST /identity/v3/projects => generated 363 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:46.580237 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b087cb-f881-4e06-ae85-9e235d959d9a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:46.676825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e322419-badc-4443-9052-0300713c7e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:46.680116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322419-badc-4443-9052-0300713c7e70 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:46.680350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322419-badc-4443-9052-0300713c7e70 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:46.680712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e322419-badc-4443-9052-0300713c7e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:46.748659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b087cb-f881-4e06-ae85-9e235d959d9a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:46.765161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba0a9a-201d-4b62-b2a8-0f941c4c3c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:46.766116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba0a9a-201d-4b62-b2a8-0f941c4c3c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:46.766603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba0a9a-201d-4b62-b2a8-0f941c4c3c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:46.767035 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba0a9a-201d-4b62-b2a8-0f941c4c3c3f admin admin] PATH_INFO: `/v3/projects/3c5b07104b3e4d96b2391cbda2e72a32/users/d727e4a4e9f045d3a7eca8e3a3a5be2a/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:46.815510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba0a9a-201d-4b62-b2a8-0f941c4c3c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5b07104b3e4d96b2391cbda2e72a32, user_id=d727e4a4e9f045d3a7eca8e3a3a5be2a, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:46.818671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba0a9a-201d-4b62-b2a8-0f941c4c3c3f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:46.845798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e322419-badc-4443-9052-0300713c7e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:46.851542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e322419-badc-4443-9052-0300713c7e70 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:46.865903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 880/1719] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:46.868107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feb389-d23a-4a61-8d58-a8741544334e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:46.870423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feb389-d23a-4a61-8d58-a8741544334e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:46.870968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42feb389-d23a-4a61-8d58-a8741544334e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:46.947581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-42feb389-d23a-4a61-8d58-a8741544334e None None] MFA Rules not processed for user `a07f995e1b17487d91a3ab109a7cd926`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:46.951382 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-42feb389-d23a-4a61-8d58-a8741544334e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:47.007379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 840/1720] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:46 2022] PUT /identity/v3/projects/3c5b07104b3e4d96b2391cbda2e72a32/users/d727e4a4e9f045d3a7eca8e3a3a5be2a/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:47.015490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba0a9a-201d-4b62-b2a8-0f941c4c3c3f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:47.159694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 881/1721] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:47.161912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feb389-d23a-4a61-8d58-a8741544334e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:47.170719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:47.183858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42feb389-d23a-4a61-8d58-a8741544334e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:47.250594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba0a9a-201d-4b62-b2a8-0f941c4c3c3f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:47.255046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:47.267542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ce10f-a585-4f56-bbd3-6f526801dfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:47.272009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ce10f-a585-4f56-bbd3-6f526801dfe1 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:47.273187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ce10f-a585-4f56-bbd3-6f526801dfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:47.273640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ce10f-a585-4f56-bbd3-6f526801dfe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:47.274907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ce10f-a585-4f56-bbd3-6f526801dfe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:47.279551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ce10f-a585-4f56-bbd3-6f526801dfe1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:47.297450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a9e36-5480-4eb9-b542-e21837e7c1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:47.298165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a9e36-5480-4eb9-b542-e21837e7c1db admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:47.298452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a9e36-5480-4eb9-b542-e21837e7c1db admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:47.298727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a9e36-5480-4eb9-b542-e21837e7c1db admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:47.301010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a9e36-5480-4eb9-b542-e21837e7c1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:47.311190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a9e36-5480-4eb9-b542-e21837e7c1db admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:47.313072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 841/1722] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:47.314584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688cdf8-faba-44bb-8958-3a61d40dfd64 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:47.314869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688cdf8-faba-44bb-8958-3a61d40dfd64 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:47.315147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688cdf8-faba-44bb-8958-3a61d40dfd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:47.341165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 882/1723] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:47.347731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a9e36-5480-4eb9-b542-e21837e7c1db admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:47.374653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a688cdf8-faba-44bb-8958-3a61d40dfd64 None None] MFA Rules not processed for user `0e8d03fe51534f8e867d0f83c3db2d36`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:47.376278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a688cdf8-faba-44bb-8958-3a61d40dfd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:47.386496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a9e36-5480-4eb9-b542-e21837e7c1db admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:47.397806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69a2d6-fdf0-4d3e-8e9a-2fe987f8d8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:47.398357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69a2d6-fdf0-4d3e-8e9a-2fe987f8d8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:47.398643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69a2d6-fdf0-4d3e-8e9a-2fe987f8d8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:47.398922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69a2d6-fdf0-4d3e-8e9a-2fe987f8d8b0 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:47.399586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69a2d6-fdf0-4d3e-8e9a-2fe987f8d8b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:47.401641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69a2d6-fdf0-4d3e-8e9a-2fe987f8d8b0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:47.503999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 842/1724] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:47.507107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7a60f-fa47-4357-a2db-916be13d850d None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:47.507451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7a60f-fa47-4357-a2db-916be13d850d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:47.507693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7a60f-fa47-4357-a2db-916be13d850d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:47.545496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 883/1725] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:47 2022] POST /identity/v3/users => generated 282 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:47.545496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69a2d6-fdf0-4d3e-8e9a-2fe987f8d8b0 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:47.578797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69a2d6-fdf0-4d3e-8e9a-2fe987f8d8b0 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:47.594326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e449319-bbd0-48b7-9d2b-1cf7a9332251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:47.598036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e449319-bbd0-48b7-9d2b-1cf7a9332251 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:47.598415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e449319-bbd0-48b7-9d2b-1cf7a9332251 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:47.598662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e449319-bbd0-48b7-9d2b-1cf7a9332251 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:47.599847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e449319-bbd0-48b7-9d2b-1cf7a9332251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:47.602967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e449319-bbd0-48b7-9d2b-1cf7a9332251 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:47.606266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-2bb7a60f-fa47-4357-a2db-916be13d850d None None] MFA Rules not processed for user `d727e4a4e9f045d3a7eca8e3a3a5be2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:47.608110 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2bb7a60f-fa47-4357-a2db-916be13d850d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:47.729568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 843/1726] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:47.731214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7a60f-fa47-4357-a2db-916be13d850d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:47.751005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7a60f-fa47-4357-a2db-916be13d850d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:47.758230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009748f-6698-4bc8-999d-e3800133f77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:47.758974 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009748f-6698-4bc8-999d-e3800133f77e admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:47.759330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009748f-6698-4bc8-999d-e3800133f77e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:47.759682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009748f-6698-4bc8-999d-e3800133f77e admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:47.761481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009748f-6698-4bc8-999d-e3800133f77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:47.762891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009748f-6698-4bc8-999d-e3800133f77e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:47.790677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 844/1727] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:47.792018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009748f-6698-4bc8-999d-e3800133f77e admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:47.799025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 884/1728] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:47 2022] POST /identity/v3/projects => generated 385 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:47.799025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e449319-bbd0-48b7-9d2b-1cf7a9332251 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:47.802812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9e449319-bbd0-48b7-9d2b-1cf7a9332251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:47.812520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009748f-6698-4bc8-999d-e3800133f77e admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:48.016921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e449319-bbd0-48b7-9d2b-1cf7a9332251 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:48.024888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d06f0-590d-4475-97cb-5641fde11ff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:48.025663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d06f0-590d-4475-97cb-5641fde11ff9 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.026102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d06f0-590d-4475-97cb-5641fde11ff9 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.027303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d06f0-590d-4475-97cb-5641fde11ff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.029338 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5860ef6-ff90-441f-9f64-d0a854b57d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:48.030544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5860ef6-ff90-441f-9f64-d0a854b57d4f admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.031058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5860ef6-ff90-441f-9f64-d0a854b57d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.031681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5860ef6-ff90-441f-9f64-d0a854b57d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.035373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5860ef6-ff90-441f-9f64-d0a854b57d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:48.045794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5860ef6-ff90-441f-9f64-d0a854b57d4f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:48.064584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 845/1729] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:48.065894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5860ef6-ff90-441f-9f64-d0a854b57d4f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:48.081275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5860ef6-ff90-441f-9f64-d0a854b57d4f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:48.088149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e6ff2-a8c0-4e00-a72d-d28d11d28dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:48.089208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e6ff2-a8c0-4e00-a72d-d28d11d28dae admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.089675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e6ff2-a8c0-4e00-a72d-d28d11d28dae admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.090136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e6ff2-a8c0-4e00-a72d-d28d11d28dae admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.091261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e6ff2-a8c0-4e00-a72d-d28d11d28dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:48.093432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e6ff2-a8c0-4e00-a72d-d28d11d28dae admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:48.110794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 846/1730] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:48.112295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e6ff2-a8c0-4e00-a72d-d28d11d28dae admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:48.134094 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e6ff2-a8c0-4e00-a72d-d28d11d28dae admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:48.154963 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14a851-5fea-4744-b756-fce8973114cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:48.158512 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a851-5fea-4744-b756-fce8973114cb admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.159622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a851-5fea-4744-b756-fce8973114cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.160096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a851-5fea-4744-b756-fce8973114cb admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.161629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14a851-5fea-4744-b756-fce8973114cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:48.168653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14a851-5fea-4744-b756-fce8973114cb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:48.186307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d06f0-590d-4475-97cb-5641fde11ff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:48.191155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d06f0-590d-4475-97cb-5641fde11ff9 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:48.227663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 885/1731] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:48.229018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d06f0-590d-4475-97cb-5641fde11ff9 service nova] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:48.245290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d06f0-590d-4475-97cb-5641fde11ff9 service nova] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:48.260750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81e840-04f5-4036-84bf-296be7bc12bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:48.262380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81e840-04f5-4036-84bf-296be7bc12bd admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.262876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81e840-04f5-4036-84bf-296be7bc12bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.263386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81e840-04f5-4036-84bf-296be7bc12bd admin admin] PATH_INFO: `/v3/projects/3fb76f3967594cd1a16c9558af9501fd/users/96dd7f05388a4c09a04dd4669d2e54d1/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.313130 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81e840-04f5-4036-84bf-296be7bc12bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb76f3967594cd1a16c9558af9501fd, user_id=96dd7f05388a4c09a04dd4669d2e54d1, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:48.320422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81e840-04f5-4036-84bf-296be7bc12bd admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:48.348240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 847/1732] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:48 2022] POST /identity/v3/projects => generated 377 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:48.348240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14a851-5fea-4744-b756-fce8973114cb admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:48.453972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 886/1733] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:48 2022] PUT /identity/v3/projects/3fb76f3967594cd1a16c9558af9501fd/users/96dd7f05388a4c09a04dd4669d2e54d1/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:48.465458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2d05f-c54e-4b6a-ba8d-964d6765043c None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.465458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2d05f-c54e-4b6a-ba8d-964d6765043c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.465458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2d05f-c54e-4b6a-ba8d-964d6765043c None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.465458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 887/1734] 10.222.0.15 () {54 vars in 912 bytes} [Tue Oct 4 12:02:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:02:48.465458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2d05f-c54e-4b6a-ba8d-964d6765043c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:48.514607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14a851-5fea-4744-b756-fce8973114cb admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:48.604988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab796c6-39a5-4da5-af6f-88aa208f1d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:48.605757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab796c6-39a5-4da5-af6f-88aa208f1d2c admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.607163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab796c6-39a5-4da5-af6f-88aa208f1d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.607163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab796c6-39a5-4da5-af6f-88aa208f1d2c admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.607895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab796c6-39a5-4da5-af6f-88aa208f1d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:48.610039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab796c6-39a5-4da5-af6f-88aa208f1d2c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:48.680243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2d05f-c54e-4b6a-ba8d-964d6765043c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:48.697061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcc03e-050f-4f0c-94d4-758cfa91e836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:48.700122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcc03e-050f-4f0c-94d4-758cfa91e836 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.700704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcc03e-050f-4f0c-94d4-758cfa91e836 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.701175 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcc03e-050f-4f0c-94d4-758cfa91e836 admin admin] PATH_INFO: `/v3/projects/5993fd3d54354600aca8bf742f9be2ed/users/c6dcf85e2ed14685bb62de7a6f47a0c4/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.728921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcc03e-050f-4f0c-94d4-758cfa91e836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5993fd3d54354600aca8bf742f9be2ed, user_id=c6dcf85e2ed14685bb62de7a6f47a0c4, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:48.733915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcc03e-050f-4f0c-94d4-758cfa91e836 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:48.785142 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 848/1735] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:48 2022] POST /identity/v3/users => generated 299 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:48.785854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab796c6-39a5-4da5-af6f-88aa208f1d2c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:48.803665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab796c6-39a5-4da5-af6f-88aa208f1d2c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:48.813220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c328ed8-b15b-40b4-96c7-eb890a93202b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:48.814225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c328ed8-b15b-40b4-96c7-eb890a93202b admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.814753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c328ed8-b15b-40b4-96c7-eb890a93202b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.815202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c328ed8-b15b-40b4-96c7-eb890a93202b admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.818890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c328ed8-b15b-40b4-96c7-eb890a93202b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:48.819251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 888/1736] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:48 2022] PUT /identity/v3/projects/5993fd3d54354600aca8bf742f9be2ed/users/c6dcf85e2ed14685bb62de7a6f47a0c4/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:48.819251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcc03e-050f-4f0c-94d4-758cfa91e836 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:48.819407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c328ed8-b15b-40b4-96c7-eb890a93202b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:48.835972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 849/1737] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:48.837826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe9008-64b2-41d1-9eed-96e80c86a251 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.838399 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe9008-64b2-41d1-9eed-96e80c86a251 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.838772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe9008-64b2-41d1-9eed-96e80c86a251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.907099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-5cfe9008-64b2-41d1-9eed-96e80c86a251 None None] MFA Rules not processed for user `96dd7f05388a4c09a04dd4669d2e54d1`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:48.909856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5cfe9008-64b2-41d1-9eed-96e80c86a251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:48.964895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcc03e-050f-4f0c-94d4-758cfa91e836 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:48.971017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65006439-b581-4d85-b724-ae43e49fc5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:48.971774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65006439-b581-4d85-b724-ae43e49fc5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:48.972247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65006439-b581-4d85-b724-ae43e49fc5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:48.972552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65006439-b581-4d85-b724-ae43e49fc5f0 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:48.973419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65006439-b581-4d85-b724-ae43e49fc5f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:48.976755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65006439-b581-4d85-b724-ae43e49fc5f0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:49.086382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 850/1738] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:49.089542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65181ea0-c358-4622-8e76-75f01cb8fff1 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:49.089994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65181ea0-c358-4622-8e76-75f01cb8fff1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:49.090456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65181ea0-c358-4622-8e76-75f01cb8fff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:49.100720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 889/1739] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:48 2022] POST /identity/v3/users => generated 289 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:49.103210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65006439-b581-4d85-b724-ae43e49fc5f0 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:49.135510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65006439-b581-4d85-b724-ae43e49fc5f0 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:49.150254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5dac5-5237-406d-9a35-2aac613553b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:49.152218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5dac5-5237-406d-9a35-2aac613553b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:49.152218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5dac5-5237-406d-9a35-2aac613553b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:49.152680 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5dac5-5237-406d-9a35-2aac613553b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:49.153990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5dac5-5237-406d-9a35-2aac613553b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:49.157172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5dac5-5237-406d-9a35-2aac613553b2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:49.170616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-65181ea0-c358-4622-8e76-75f01cb8fff1 None None] MFA Rules not processed for user `073e43c049224af29e9f6d426ea47e84`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:49.171982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-65181ea0-c358-4622-8e76-75f01cb8fff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:49.179735 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 890/1740] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:49.181763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4e014-f8b2-4804-8f4b-a14a5baa13d3 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:49.182241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4e014-f8b2-4804-8f4b-a14a5baa13d3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:49.182819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4e014-f8b2-4804-8f4b-a14a5baa13d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:49.261019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c6c4e014-f8b2-4804-8f4b-a14a5baa13d3 None None] MFA Rules not processed for user `c6dcf85e2ed14685bb62de7a6f47a0c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:49.261019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c6c4e014-f8b2-4804-8f4b-a14a5baa13d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:49.273921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 851/1741] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:02:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:49.275526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65181ea0-c358-4622-8e76-75f01cb8fff1 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:49.308716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:49.311675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65181ea0-c358-4622-8e76-75f01cb8fff1 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:49.330044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee552b0-c638-4998-8945-ed574950b811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:49.331339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee552b0-c638-4998-8945-ed574950b811 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:49.331922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee552b0-c638-4998-8945-ed574950b811 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:49.332502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee552b0-c638-4998-8945-ed574950b811 admin admin] PATH_INFO: `/v3/projects/5e83563c63f44551bca1c6debe26340e/users/c1592d75faf54feebedca83d35e3b971/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:49.381796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee552b0-c638-4998-8945-ed574950b811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e83563c63f44551bca1c6debe26340e, user_id=c1592d75faf54feebedca83d35e3b971, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:49.390154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee552b0-c638-4998-8945-ed574950b811 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:49.470245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 891/1742] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:49.470245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4e014-f8b2-4804-8f4b-a14a5baa13d3 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:49.494514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 852/1743] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:49 2022] PUT /identity/v3/projects/5e83563c63f44551bca1c6debe26340e/users/c1592d75faf54feebedca83d35e3b971/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:49.498911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee552b0-c638-4998-8945-ed574950b811 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:49.507633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4e014-f8b2-4804-8f4b-a14a5baa13d3 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:49.650011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdb864-c633-4a7a-b696-80abe193fc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:49.673093 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdb864-c633-4a7a-b696-80abe193fc20 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:49.673093 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdb864-c633-4a7a-b696-80abe193fc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:49.673093 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdb864-c633-4a7a-b696-80abe193fc20 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:49.673093 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdb864-c633-4a7a-b696-80abe193fc20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:49.676648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdb864-c633-4a7a-b696-80abe193fc20 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:49.680614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee552b0-c638-4998-8945-ed574950b811 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:49.690057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45ec86-02ba-4027-b71d-b0aff1f9491f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:49.691315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45ec86-02ba-4027-b71d-b0aff1f9491f admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:49.691735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45ec86-02ba-4027-b71d-b0aff1f9491f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:49.692328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45ec86-02ba-4027-b71d-b0aff1f9491f admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:49.693719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45ec86-02ba-4027-b71d-b0aff1f9491f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:49.699178 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45ec86-02ba-4027-b71d-b0aff1f9491f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:49.722012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 853/1744] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:49.723927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9cbc8-89f0-4d4f-aff1-53b7bf0340d6 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:49.730396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9cbc8-89f0-4d4f-aff1-53b7bf0340d6 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:49.730396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9cbc8-89f0-4d4f-aff1-53b7bf0340d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:49.752791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-63e9cbc8-89f0-4d4f-aff1-53b7bf0340d6 None None] MFA Rules not processed for user `fd040e13fd6c4b0dbea6a65932f84cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:49.754371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-63e9cbc8-89f0-4d4f-aff1-53b7bf0340d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:49.824360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:49.853641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 854/1745] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:49.853641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9cbc8-89f0-4d4f-aff1-53b7bf0340d6 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:49.878460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9cbc8-89f0-4d4f-aff1-53b7bf0340d6 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:49.892429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 892/1746] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:49 2022] POST /identity/v3/projects => generated 363 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:49.893533 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdb864-c633-4a7a-b696-80abe193fc20 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:49.895631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-77cdb864-c633-4a7a-b696-80abe193fc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:49.906997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d938a6-457e-435c-932c-3eb83a71b058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:49.907706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d938a6-457e-435c-932c-3eb83a71b058 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:49.908486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d938a6-457e-435c-932c-3eb83a71b058 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:49.908786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d938a6-457e-435c-932c-3eb83a71b058 admin admin] PATH_INFO: `/v3/projects/7947ad8bf168403c830f8490364e9004/users/ee8c303d966844d49909b14d85093d1d/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:49.963091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d938a6-457e-435c-932c-3eb83a71b058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7947ad8bf168403c830f8490364e9004, user_id=ee8c303d966844d49909b14d85093d1d, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:49.982099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d938a6-457e-435c-932c-3eb83a71b058 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:50.098069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdb864-c633-4a7a-b696-80abe193fc20 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:50.152656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 855/1747] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:49 2022] PUT /identity/v3/projects/7947ad8bf168403c830f8490364e9004/users/ee8c303d966844d49909b14d85093d1d/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:50.152656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d938a6-457e-435c-932c-3eb83a71b058 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:50.158064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf1d84-339a-4cbc-a7e7-d9117bc9bb7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:50.163648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf1d84-339a-4cbc-a7e7-d9117bc9bb7f service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:50.166527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf1d84-339a-4cbc-a7e7-d9117bc9bb7f service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:50.168401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf1d84-339a-4cbc-a7e7-d9117bc9bb7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:50.234494 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf1d84-339a-4cbc-a7e7-d9117bc9bb7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:50.241128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf1d84-339a-4cbc-a7e7-d9117bc9bb7f service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:50.275691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 893/1748] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:50.282508 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e690f8d-49d6-4ba5-a5f2-08dd54687b09 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:50.282990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e690f8d-49d6-4ba5-a5f2-08dd54687b09 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:50.283565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e690f8d-49d6-4ba5-a5f2-08dd54687b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:50.420308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9e690f8d-49d6-4ba5-a5f2-08dd54687b09 None None] MFA Rules not processed for user `c1592d75faf54feebedca83d35e3b971`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:50.429744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9e690f8d-49d6-4ba5-a5f2-08dd54687b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:50.484577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d938a6-457e-435c-932c-3eb83a71b058 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:50.514440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cf704-fa7c-4447-8cf6-a8392c171303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:50.518699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cf704-fa7c-4447-8cf6-a8392c171303 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:50.520679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cf704-fa7c-4447-8cf6-a8392c171303 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:50.525097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cf704-fa7c-4447-8cf6-a8392c171303 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:50.526478 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cf704-fa7c-4447-8cf6-a8392c171303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:50.539803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cf704-fa7c-4447-8cf6-a8392c171303 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:50.698165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 894/1749] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:50.699846 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e690f8d-49d6-4ba5-a5f2-08dd54687b09 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:50.733674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 856/1750] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:50 2022] POST /identity/v3/projects => generated 367 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:50.736193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cf704-fa7c-4447-8cf6-a8392c171303 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:50.763455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e690f8d-49d6-4ba5-a5f2-08dd54687b09 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:50.931981 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cf704-fa7c-4447-8cf6-a8392c171303 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:50.936102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2d813-238d-461c-9db9-a2d48a433490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:50.937054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d813-238d-461c-9db9-a2d48a433490 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:50.937547 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d813-238d-461c-9db9-a2d48a433490 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:50.938087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d813-238d-461c-9db9-a2d48a433490 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:50.938976 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2d813-238d-461c-9db9-a2d48a433490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:50.940125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73f475-221e-4d87-9559-c447b6bfafc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:50.941389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73f475-221e-4d87-9559-c447b6bfafc8 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:50.941753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73f475-221e-4d87-9559-c447b6bfafc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:50.942103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73f475-221e-4d87-9559-c447b6bfafc8 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:50.942809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2d813-238d-461c-9db9-a2d48a433490 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:50.943466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73f475-221e-4d87-9559-c447b6bfafc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:50.946048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae73f475-221e-4d87-9559-c447b6bfafc8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:50.967354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 895/1751] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:50.968800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2d813-238d-461c-9db9-a2d48a433490 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:51.042388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2d813-238d-461c-9db9-a2d48a433490 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:51.062517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e98ea6-ac7c-45cd-88f4-d27f56f4e131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:51.063139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e98ea6-ac7c-45cd-88f4-d27f56f4e131 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:51.063434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e98ea6-ac7c-45cd-88f4-d27f56f4e131 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:51.063713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e98ea6-ac7c-45cd-88f4-d27f56f4e131 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:51.064816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e98ea6-ac7c-45cd-88f4-d27f56f4e131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:51.070222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e98ea6-ac7c-45cd-88f4-d27f56f4e131 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:51.079257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 896/1752] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:51.080239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e98ea6-ac7c-45cd-88f4-d27f56f4e131 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:51.091878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e98ea6-ac7c-45cd-88f4-d27f56f4e131 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:51.098864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877593e-5a15-43af-8e7f-12032f015ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:51.100311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877593e-5a15-43af-8e7f-12032f015ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:51.100721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877593e-5a15-43af-8e7f-12032f015ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:51.101011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877593e-5a15-43af-8e7f-12032f015ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:51.101694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877593e-5a15-43af-8e7f-12032f015ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:51.104268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877593e-5a15-43af-8e7f-12032f015ce0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:51.183584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 857/1753] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:50 2022] POST /identity/v3/users => generated 288 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:51.187457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73f475-221e-4d87-9559-c447b6bfafc8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:51.216280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73f475-221e-4d87-9559-c447b6bfafc8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:51.237482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49867b-31f2-4cd9-bfa1-ad0fb1f7d9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:51.239027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49867b-31f2-4cd9-bfa1-ad0fb1f7d9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:51.239322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49867b-31f2-4cd9-bfa1-ad0fb1f7d9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:51.240132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49867b-31f2-4cd9-bfa1-ad0fb1f7d9d5 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:51.241157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49867b-31f2-4cd9-bfa1-ad0fb1f7d9d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:51.251713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49867b-31f2-4cd9-bfa1-ad0fb1f7d9d5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:51.286158 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 897/1754] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:51 2022] POST /identity/v3/projects => generated 365 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:51.289424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1d811-e2ec-4568-855a-b82da4b1b126 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:51.289997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1d811-e2ec-4568-855a-b82da4b1b126 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:51.290606 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1d811-e2ec-4568-855a-b82da4b1b126 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:51.293961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 898/1755] 10.222.0.15 () {54 vars in 912 bytes} [Tue Oct 4 12:02:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:02:51.295521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1d811-e2ec-4568-855a-b82da4b1b126 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:51.426112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 858/1756] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:51 2022] POST /identity/v3/users => generated 290 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:51.431351 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49867b-31f2-4cd9-bfa1-ad0fb1f7d9d5 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:51.449550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1d811-e2ec-4568-855a-b82da4b1b126 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:51.470241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ff8b0-f1da-49cf-8c45-56b7f7c39118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:51.472711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ff8b0-f1da-49cf-8c45-56b7f7c39118 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:51.477422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ff8b0-f1da-49cf-8c45-56b7f7c39118 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:51.478323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ff8b0-f1da-49cf-8c45-56b7f7c39118 admin admin] PATH_INFO: `/v3/projects/fcf3e2e343e04e0888c912a1ef9d9f61/users/3734d8a1452f4e28a48e747f46d98b29/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:51.479261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49867b-31f2-4cd9-bfa1-ad0fb1f7d9d5 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:51.494446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38958e-4560-4ff9-b6a4-cf9e2a664a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:51.495585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38958e-4560-4ff9-b6a4-cf9e2a664a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:51.496783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38958e-4560-4ff9-b6a4-cf9e2a664a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:51.497339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38958e-4560-4ff9-b6a4-cf9e2a664a2a admin admin] PATH_INFO: `/v3/projects/7947ad8bf168403c830f8490364e9004/users/ee8c303d966844d49909b14d85093d1d/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:51.506179 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38958e-4560-4ff9-b6a4-cf9e2a664a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7947ad8bf168403c830f8490364e9004, user_id=ee8c303d966844d49909b14d85093d1d, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:51.518155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38958e-4560-4ff9-b6a4-cf9e2a664a2a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:51.537808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ff8b0-f1da-49cf-8c45-56b7f7c39118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf3e2e343e04e0888c912a1ef9d9f61, user_id=3734d8a1452f4e28a48e747f46d98b29, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:51.545716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ff8b0-f1da-49cf-8c45-56b7f7c39118 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:51.554096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 859/1757] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:51 2022] PUT /identity/v3/projects/7947ad8bf168403c830f8490364e9004/users/ee8c303d966844d49909b14d85093d1d/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:51.555517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38958e-4560-4ff9-b6a4-cf9e2a664a2a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:51.657653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:51.681979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38958e-4560-4ff9-b6a4-cf9e2a664a2a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:51.690838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d08f3-484e-40c8-bdd2-d0a98faae043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:51.692256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d08f3-484e-40c8-bdd2-d0a98faae043 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:51.692870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d08f3-484e-40c8-bdd2-d0a98faae043 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:51.694073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d08f3-484e-40c8-bdd2-d0a98faae043 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:51.695311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d08f3-484e-40c8-bdd2-d0a98faae043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:51.698725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d08f3-484e-40c8-bdd2-d0a98faae043 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:51.719784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 899/1758] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:51 2022] PUT /identity/v3/projects/fcf3e2e343e04e0888c912a1ef9d9f61/users/3734d8a1452f4e28a48e747f46d98b29/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:51.721458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ff8b0-f1da-49cf-8c45-56b7f7c39118 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:51.734460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 860/1759] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:51.741199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04070dbc-b615-45f6-a5d4-084325560b7e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:51.745005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04070dbc-b615-45f6-a5d4-084325560b7e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:51.745350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04070dbc-b615-45f6-a5d4-084325560b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:51.787372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-04070dbc-b615-45f6-a5d4-084325560b7e None None] MFA Rules not processed for user `073e43c049224af29e9f6d426ea47e84`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:51.787372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-04070dbc-b615-45f6-a5d4-084325560b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:51.875523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ff8b0-f1da-49cf-8c45-56b7f7c39118 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:51.880970 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd8f28-d6e9-422c-99b6-bbf31e101081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:51.881883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd8f28-d6e9-422c-99b6-bbf31e101081 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:51.882279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd8f28-d6e9-422c-99b6-bbf31e101081 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:51.882677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd8f28-d6e9-422c-99b6-bbf31e101081 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:51.883549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd8f28-d6e9-422c-99b6-bbf31e101081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:51.885695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd8f28-d6e9-422c-99b6-bbf31e101081 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:51.901569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 861/1760] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:02:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:51.903224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04070dbc-b615-45f6-a5d4-084325560b7e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:51.918318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04070dbc-b615-45f6-a5d4-084325560b7e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:51.924460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934708a0-fbad-4f3e-96f3-860d538650d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:51.925217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934708a0-fbad-4f3e-96f3-860d538650d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:51.925597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934708a0-fbad-4f3e-96f3-860d538650d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:51.926323 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934708a0-fbad-4f3e-96f3-860d538650d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:51.926771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934708a0-fbad-4f3e-96f3-860d538650d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:51.928951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934708a0-fbad-4f3e-96f3-860d538650d5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:51.941197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 862/1761] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:51.942080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934708a0-fbad-4f3e-96f3-860d538650d5 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:51.943904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-934708a0-fbad-4f3e-96f3-860d538650d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:52.082250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934708a0-fbad-4f3e-96f3-860d538650d5 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:52.092548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179361d-8b32-4bf7-9f40-c22bf88efce9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:52.093582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179361d-8b32-4bf7-9f40-c22bf88efce9 service placement] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:52.094070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179361d-8b32-4bf7-9f40-c22bf88efce9 service placement] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:52.094569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179361d-8b32-4bf7-9f40-c22bf88efce9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:52.097705 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-6179361d-8b32-4bf7-9f40-c22bf88efce9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:52.123493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179361d-8b32-4bf7-9f40-c22bf88efce9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:52.125350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179361d-8b32-4bf7-9f40-c22bf88efce9 service placement] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:52.135887 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 900/1762] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:51 2022] POST /identity/v3/users => generated 289 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:52.138732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206d5c-6046-4a7f-8772-8110b4434852 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:52.139230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206d5c-6046-4a7f-8772-8110b4434852 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:52.139620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206d5c-6046-4a7f-8772-8110b4434852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:52.148886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 863/1763] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:52.149180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179361d-8b32-4bf7-9f40-c22bf88efce9 service placement] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:52.162547 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:52.173962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179361d-8b32-4bf7-9f40-c22bf88efce9 service placement] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:52.199803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00f134-5535-42dd-be36-e69ef2c1e3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:52.199803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00f134-5535-42dd-be36-e69ef2c1e3f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:52.199803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00f134-5535-42dd-be36-e69ef2c1e3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:52.199803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00f134-5535-42dd-be36-e69ef2c1e3f0 admin admin] PATH_INFO: `/v3/projects/8cd42695272b495ca48a7f9608ef1c9f/users/ba4d5680fbdb4ad9b4ad0e699067b508/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:52.225158 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d8206d5c-6046-4a7f-8772-8110b4434852 None None] MFA Rules not processed for user `ee8c303d966844d49909b14d85093d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:52.226492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d8206d5c-6046-4a7f-8772-8110b4434852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:52.242825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c00f134-5535-42dd-be36-e69ef2c1e3f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd42695272b495ca48a7f9608ef1c9f, user_id=ba4d5680fbdb4ad9b4ad0e699067b508, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:52.248607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c00f134-5535-42dd-be36-e69ef2c1e3f0 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:52.397845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 901/1764] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:52.403129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eb2e2-9ad0-47ab-a3e9-c9baaacf9c16 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:52.403129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eb2e2-9ad0-47ab-a3e9-c9baaacf9c16 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:52.403129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eb2e2-9ad0-47ab-a3e9-c9baaacf9c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:52.435075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 864/1765] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:52 2022] PUT /identity/v3/projects/8cd42695272b495ca48a7f9608ef1c9f/users/ba4d5680fbdb4ad9b4ad0e699067b508/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:52.435932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00f134-5535-42dd-be36-e69ef2c1e3f0 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:52.449908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8c00f134-5535-42dd-be36-e69ef2c1e3f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:52.522410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9a3eb2e2-9ad0-47ab-a3e9-c9baaacf9c16 None None] MFA Rules not processed for user `3734d8a1452f4e28a48e747f46d98b29`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:52.546398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9a3eb2e2-9ad0-47ab-a3e9-c9baaacf9c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:52.594411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00f134-5535-42dd-be36-e69ef2c1e3f0 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:52.601317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f95ad-1d17-4fe3-9c9f-9308e30fe5dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:52.601941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f95ad-1d17-4fe3-9c9f-9308e30fe5dd service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:52.602299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f95ad-1d17-4fe3-9c9f-9308e30fe5dd service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:52.602634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f95ad-1d17-4fe3-9c9f-9308e30fe5dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:52.613393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f95ad-1d17-4fe3-9c9f-9308e30fe5dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:52.618565 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f95ad-1d17-4fe3-9c9f-9308e30fe5dd service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:52.641865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 865/1766] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:52.642856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f95ad-1d17-4fe3-9c9f-9308e30fe5dd service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:52.841197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 902/1767] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:52.841197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eb2e2-9ad0-47ab-a3e9-c9baaacf9c16 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:53.037557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f95ad-1d17-4fe3-9c9f-9308e30fe5dd service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:53.077626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eb2e2-9ad0-47ab-a3e9-c9baaacf9c16 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:53.087188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733ce21-63ab-4817-a100-8aec5cd4530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:53.087881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733ce21-63ab-4817-a100-8aec5cd4530e admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:53.088160 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733ce21-63ab-4817-a100-8aec5cd4530e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:53.088495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733ce21-63ab-4817-a100-8aec5cd4530e admin admin] PATH_INFO: `/v3/projects/1ca16074514c4ce5853357b8c2e4a9d5/users/65339b009fe147f99df52b13c024afab/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:53.089393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c5a96-5830-41f2-ba49-19efa7961209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:53.090441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c5a96-5830-41f2-ba49-19efa7961209 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:53.091151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c5a96-5830-41f2-ba49-19efa7961209 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:53.091688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c5a96-5830-41f2-ba49-19efa7961209 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:53.092919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c5a96-5830-41f2-ba49-19efa7961209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:53.096646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c5a96-5830-41f2-ba49-19efa7961209 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:53.123303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733ce21-63ab-4817-a100-8aec5cd4530e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca16074514c4ce5853357b8c2e4a9d5, user_id=65339b009fe147f99df52b13c024afab, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:53.127804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733ce21-63ab-4817-a100-8aec5cd4530e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:53.133865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 903/1768] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:53.135550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c5a96-5830-41f2-ba49-19efa7961209 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:53.140473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ff0c5a96-5830-41f2-ba49-19efa7961209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:53.248794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c5a96-5830-41f2-ba49-19efa7961209 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:53.259617 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15355b09-81a9-4d7f-9bfc-291b83bb14b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:53.260641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355b09-81a9-4d7f-9bfc-291b83bb14b0 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:53.261101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355b09-81a9-4d7f-9bfc-291b83bb14b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:53.261557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355b09-81a9-4d7f-9bfc-291b83bb14b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:53.310502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 866/1769] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:52 2022] PUT /identity/v3/projects/1ca16074514c4ce5853357b8c2e4a9d5/users/65339b009fe147f99df52b13c024afab/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:53.312832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733ce21-63ab-4817-a100-8aec5cd4530e admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:53.376839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15355b09-81a9-4d7f-9bfc-291b83bb14b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:53.379025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15355b09-81a9-4d7f-9bfc-291b83bb14b0 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:53.399491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 904/1770] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:53.402214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15355b09-81a9-4d7f-9bfc-291b83bb14b0 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:53.421802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15355b09-81a9-4d7f-9bfc-291b83bb14b0 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:53.514683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:53.535185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733ce21-63ab-4817-a100-8aec5cd4530e admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:53.541539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a0554-9592-401b-b9db-7c49605b5ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:53.542180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a0554-9592-401b-b9db-7c49605b5ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:53.542493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a0554-9592-401b-b9db-7c49605b5ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:53.542732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a0554-9592-401b-b9db-7c49605b5ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:53.546927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a0554-9592-401b-b9db-7c49605b5ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:53.548686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a0554-9592-401b-b9db-7c49605b5ef8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:53.590365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 867/1771] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:53.594269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a32534-de63-40db-930f-be3bb1615bcf None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:53.595071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a32534-de63-40db-930f-be3bb1615bcf None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:53.595587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a32534-de63-40db-930f-be3bb1615bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:53.634069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-30a32534-de63-40db-930f-be3bb1615bcf None None] MFA Rules not processed for user `0e8d03fe51534f8e867d0f83c3db2d36`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:53.635699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-30a32534-de63-40db-930f-be3bb1615bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:53.648606 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdff3c-5f17-493b-9376-324f6c7c38ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:53.652578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdff3c-5f17-493b-9376-324f6c7c38ca admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:53.652578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdff3c-5f17-493b-9376-324f6c7c38ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:53.652578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdff3c-5f17-493b-9376-324f6c7c38ca admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:53.652578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdff3c-5f17-493b-9376-324f6c7c38ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:53.656448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdff3c-5f17-493b-9376-324f6c7c38ca admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:53.814821 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 905/1772] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:02:53 2022] POST /identity/v3/projects => generated 379 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:53.814821 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdff3c-5f17-493b-9376-324f6c7c38ca admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:53.819084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 868/1773] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:53.827489 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a32534-de63-40db-930f-be3bb1615bcf None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:53.857623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a32534-de63-40db-930f-be3bb1615bcf None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:53.974648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdff3c-5f17-493b-9376-324f6c7c38ca admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:53.980919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee567c-171e-40af-9ee7-b90b62ab2b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:53.981739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee567c-171e-40af-9ee7-b90b62ab2b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:53.982109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee567c-171e-40af-9ee7-b90b62ab2b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:53.990192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee567c-171e-40af-9ee7-b90b62ab2b15 admin admin] PATH_INFO: `/v3/projects/242b49781f10432c879829f777cfe789/users/9585a44aa90b4218ae69fd4700aeafca/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:54.051352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca0fbf-a145-4dae-9b5d-301299a12656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:54.052482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca0fbf-a145-4dae-9b5d-301299a12656 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:54.053511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca0fbf-a145-4dae-9b5d-301299a12656 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:54.054917 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca0fbf-a145-4dae-9b5d-301299a12656 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:54.056898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca0fbf-a145-4dae-9b5d-301299a12656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:54.060541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca0fbf-a145-4dae-9b5d-301299a12656 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:54.074676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee567c-171e-40af-9ee7-b90b62ab2b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=242b49781f10432c879829f777cfe789, user_id=9585a44aa90b4218ae69fd4700aeafca, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:54.080438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee567c-171e-40af-9ee7-b90b62ab2b15 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:54.080891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:54.155421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 869/1774] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:54.157670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca0fbf-a145-4dae-9b5d-301299a12656 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:54.192496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca0fbf-a145-4dae-9b5d-301299a12656 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:54.197807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:54.198618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:54.199111 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:54.199249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] PATH_INFO: `/v3/users/a07f995e1b17487d91a3ab109a7cd926` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:54.200785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07f995e1b17487d91a3ab109a7cd926)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:54.202887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:54.247439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 906/1775] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:53 2022] PUT /identity/v3/projects/242b49781f10432c879829f777cfe789/users/9585a44aa90b4218ae69fd4700aeafca/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:54.248949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee567c-171e-40af-9ee7-b90b62ab2b15 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:54.413452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee567c-171e-40af-9ee7-b90b62ab2b15 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:54.420472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000eac4-b29e-46a7-81d1-daa590fda57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:54.421207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000eac4-b29e-46a7-81d1-daa590fda57a admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:54.421580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000eac4-b29e-46a7-81d1-daa590fda57a admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:54.421926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000eac4-b29e-46a7-81d1-daa590fda57a admin admin] PATH_INFO: `/v3/projects/8cd42695272b495ca48a7f9608ef1c9f/users/ba4d5680fbdb4ad9b4ad0e699067b508/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:54.424335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000eac4-b29e-46a7-81d1-daa590fda57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd42695272b495ca48a7f9608ef1c9f, user_id=ba4d5680fbdb4ad9b4ad0e699067b508, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:54.429634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000eac4-b29e-46a7-81d1-daa590fda57a admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:54.446251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 907/1776] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:54 2022] PUT /identity/v3/projects/8cd42695272b495ca48a7f9608ef1c9f/users/ba4d5680fbdb4ad9b4ad0e699067b508/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:54.447181 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000eac4-b29e-46a7-81d1-daa590fda57a admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:54.479611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07f995e1b17487d91a3ab109a7cd926'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:54.496647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07f995e1b17487d91a3ab109a7cd926'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:54.499978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07f995e1b17487d91a3ab109a7cd926'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:54.636828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000eac4-b29e-46a7-81d1-daa590fda57a admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:54.644516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07f995e1b17487d91a3ab109a7cd926'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:54.668953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bfbbb-0c27-462e-9fb1-4481ee917898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:54.671149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bfbbb-0c27-462e-9fb1-4481ee917898 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:54.672982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bfbbb-0c27-462e-9fb1-4481ee917898 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:54.673298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bfbbb-0c27-462e-9fb1-4481ee917898 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:54.674006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bfbbb-0c27-462e-9fb1-4481ee917898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:54.679850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07f995e1b17487d91a3ab109a7cd926'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:54.680993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bfbbb-0c27-462e-9fb1-4481ee917898 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:54.693251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 870/1777] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:54 2022] DELETE /identity/v3/users/a07f995e1b17487d91a3ab109a7cd926 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:54.694384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:54.701235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:54.846647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 908/1778] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:02:54 2022] POST /identity/v3/users => generated 296 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:02:54.847805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bfbbb-0c27-462e-9fb1-4481ee917898 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:54.852705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-272bfbbb-0c27-462e-9fb1-4481ee917898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:54.856887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf948dd-8700-418a-96e0-d67943f808cd admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:54.873875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff96a2-185e-4da4-b959-d518825adde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:54.875726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff96a2-185e-4da4-b959-d518825adde8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:54.877079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff96a2-185e-4da4-b959-d518825adde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:54.877816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff96a2-185e-4da4-b959-d518825adde8 admin admin] PATH_INFO: `/v3/projects/1ca16074514c4ce5853357b8c2e4a9d5/users/65339b009fe147f99df52b13c024afab/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:54.882173 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff96a2-185e-4da4-b959-d518825adde8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca16074514c4ce5853357b8c2e4a9d5, user_id=65339b009fe147f99df52b13c024afab, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:54.888652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff96a2-185e-4da4-b959-d518825adde8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:54.890820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bfbbb-0c27-462e-9fb1-4481ee917898 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:54.910420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abdfd4-b376-4423-b4fc-ba5b2d5e89a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:54.911085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abdfd4-b376-4423-b4fc-ba5b2d5e89a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:54.911955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abdfd4-b376-4423-b4fc-ba5b2d5e89a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:54.912295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abdfd4-b376-4423-b4fc-ba5b2d5e89a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:54.912917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abdfd4-b376-4423-b4fc-ba5b2d5e89a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:54.914566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abdfd4-b376-4423-b4fc-ba5b2d5e89a0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:54.925132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 871/1779] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:54 2022] PUT /identity/v3/projects/1ca16074514c4ce5853357b8c2e4a9d5/users/65339b009fe147f99df52b13c024afab/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:54.940595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73c2f0-ab49-4a25-8dfc-83e9932ba245 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:54.940595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73c2f0-ab49-4a25-8dfc-83e9932ba245 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:54.940595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73c2f0-ab49-4a25-8dfc-83e9932ba245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:54.941101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 909/1780] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:54.941101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb5366-181d-4700-8d06-bff72f102816 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:54.941101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb5366-181d-4700-8d06-bff72f102816 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:54.941101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb5366-181d-4700-8d06-bff72f102816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:54.957839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-cfeb5366-181d-4700-8d06-bff72f102816 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:54.959767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cfeb5366-181d-4700-8d06-bff72f102816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:55.001862 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-5f73c2f0-ab49-4a25-8dfc-83e9932ba245 None None] MFA Rules not processed for user `ba4d5680fbdb4ad9b4ad0e699067b508`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:55.003785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5f73c2f0-ab49-4a25-8dfc-83e9932ba245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:55.119932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 910/1781] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:55.120783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb5366-181d-4700-8d06-bff72f102816 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:55.125337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cfeb5366-181d-4700-8d06-bff72f102816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:55.138889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb5366-181d-4700-8d06-bff72f102816 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:55.147678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad4797-2ffd-4181-8b17-f80ace35f0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:55.150199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad4797-2ffd-4181-8b17-f80ace35f0aa admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:55.152873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad4797-2ffd-4181-8b17-f80ace35f0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:55.174763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad4797-2ffd-4181-8b17-f80ace35f0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:55.174763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad4797-2ffd-4181-8b17-f80ace35f0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:55.176012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 872/1782] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:55.176012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73c2f0-ab49-4a25-8dfc-83e9932ba245 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:55.176301 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad4797-2ffd-4181-8b17-f80ace35f0aa admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:55.186901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 911/1783] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:55.186901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833755a-83cd-4422-81be-d93d4b317c49 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:55.186901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833755a-83cd-4422-81be-d93d4b317c49 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:55.186901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833755a-83cd-4422-81be-d93d4b317c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:55.200465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73c2f0-ab49-4a25-8dfc-83e9932ba245 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:55.213696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90788f4b-c89a-4dfc-a85a-d2a9ab1fb9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:55.214835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90788f4b-c89a-4dfc-a85a-d2a9ab1fb9d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:55.215240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90788f4b-c89a-4dfc-a85a-d2a9ab1fb9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:55.215618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90788f4b-c89a-4dfc-a85a-d2a9ab1fb9d8 admin admin] PATH_INFO: `/v3/projects/242b49781f10432c879829f777cfe789/users/9585a44aa90b4218ae69fd4700aeafca/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:55.220897 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90788f4b-c89a-4dfc-a85a-d2a9ab1fb9d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=242b49781f10432c879829f777cfe789, user_id=9585a44aa90b4218ae69fd4700aeafca, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:55.227650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90788f4b-c89a-4dfc-a85a-d2a9ab1fb9d8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:55.247896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 873/1784] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:55 2022] PUT /identity/v3/projects/242b49781f10432c879829f777cfe789/users/9585a44aa90b4218ae69fd4700aeafca/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:55.249526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90788f4b-c89a-4dfc-a85a-d2a9ab1fb9d8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:55.251542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-90788f4b-c89a-4dfc-a85a-d2a9ab1fb9d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:55.319488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b833755a-83cd-4422-81be-d93d4b317c49 None None] MFA Rules not processed for user `65339b009fe147f99df52b13c024afab`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:55.325479 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b833755a-83cd-4422-81be-d93d4b317c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:55.457081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90788f4b-c89a-4dfc-a85a-d2a9ab1fb9d8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:55.464919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d066-fef3-4fa3-b8e6-3e2c236d3d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:55.466069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d066-fef3-4fa3-b8e6-3e2c236d3d54 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:55.466553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d066-fef3-4fa3-b8e6-3e2c236d3d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:55.467042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d066-fef3-4fa3-b8e6-3e2c236d3d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:55.563456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 912/1785] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:55.563456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833755a-83cd-4422-81be-d93d4b317c49 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:55.619028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995d066-fef3-4fa3-b8e6-3e2c236d3d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:55.620967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995d066-fef3-4fa3-b8e6-3e2c236d3d54 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:55.633625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 874/1786] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:55.638448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d066-fef3-4fa3-b8e6-3e2c236d3d54 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:55.642168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e995d066-fef3-4fa3-b8e6-3e2c236d3d54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:55.700151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833755a-83cd-4422-81be-d93d4b317c49 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:55.721483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3378d73-7594-43ac-949d-a69a6ee2029c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:55.722745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3378d73-7594-43ac-949d-a69a6ee2029c admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:55.723211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3378d73-7594-43ac-949d-a69a6ee2029c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:55.723727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3378d73-7594-43ac-949d-a69a6ee2029c admin admin] PATH_INFO: `/v3/projects/17c5623a13f9414e9ece79219ed9eece/users/313bfa4722374b62b973f6ea9ff549ac/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:55.761155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3378d73-7594-43ac-949d-a69a6ee2029c admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c5623a13f9414e9ece79219ed9eece, user_id=313bfa4722374b62b973f6ea9ff549ac, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:55.766487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3378d73-7594-43ac-949d-a69a6ee2029c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:55.817361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d066-fef3-4fa3-b8e6-3e2c236d3d54 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:55.824245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ffc5-cde5-433c-9808-23b264766a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:55.825463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ffc5-cde5-433c-9808-23b264766a0f service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:55.826372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ffc5-cde5-433c-9808-23b264766a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:55.826900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ffc5-cde5-433c-9808-23b264766a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:55.927651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 913/1787] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:55 2022] PUT /identity/v3/projects/17c5623a13f9414e9ece79219ed9eece/users/313bfa4722374b62b973f6ea9ff549ac/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:55.935445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a330045-5553-490d-aaec-ecde722cfd9a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:55.936485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a330045-5553-490d-aaec-ecde722cfd9a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:55.937176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a330045-5553-490d-aaec-ecde722cfd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:56.039268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4a330045-5553-490d-aaec-ecde722cfd9a None None] MFA Rules not processed for user `9585a44aa90b4218ae69fd4700aeafca`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:56.041376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4a330045-5553-490d-aaec-ecde722cfd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:56.070547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434ffc5-cde5-433c-9808-23b264766a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:56.073683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434ffc5-cde5-433c-9808-23b264766a0f service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:56.092414 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 875/1788] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:56.093910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ffc5-cde5-433c-9808-23b264766a0f service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:56.099285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1434ffc5-cde5-433c-9808-23b264766a0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:56.121433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ffc5-cde5-433c-9808-23b264766a0f service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:56.212692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa37f-bd5a-491a-8dc9-ca4aaff86718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:56.213389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa37f-bd5a-491a-8dc9-ca4aaff86718 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:56.213675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa37f-bd5a-491a-8dc9-ca4aaff86718 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:56.214235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa37f-bd5a-491a-8dc9-ca4aaff86718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:56.254583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 914/1789] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:02:56.257310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a330045-5553-490d-aaec-ecde722cfd9a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:56.378522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfa37f-bd5a-491a-8dc9-ca4aaff86718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:56.382484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfa37f-bd5a-491a-8dc9-ca4aaff86718 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:56.384242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:56.413102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 876/1790] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:56.479809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa37f-bd5a-491a-8dc9-ca4aaff86718 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:56.491152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa37f-bd5a-491a-8dc9-ca4aaff86718 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:56.505225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b3eba-9923-440e-83b9-02fb1c642a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:56.506109 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b3eba-9923-440e-83b9-02fb1c642a00 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:56.506535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b3eba-9923-440e-83b9-02fb1c642a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:56.507100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b3eba-9923-440e-83b9-02fb1c642a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:56.531321 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b3eba-9923-440e-83b9-02fb1c642a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:56.537934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b3eba-9923-440e-83b9-02fb1c642a00 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:56.575207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 877/1791] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:02:56.611798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a330045-5553-490d-aaec-ecde722cfd9a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:56.617560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca63f3b-17ad-46c2-a51b-31643b8064bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:56.618327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca63f3b-17ad-46c2-a51b-31643b8064bb admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:56.618673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca63f3b-17ad-46c2-a51b-31643b8064bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:56.619017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca63f3b-17ad-46c2-a51b-31643b8064bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:56.619797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca63f3b-17ad-46c2-a51b-31643b8064bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:56.622244 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca63f3b-17ad-46c2-a51b-31643b8064bb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:56.640359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 915/1792] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:02:56.642070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b3eba-9923-440e-83b9-02fb1c642a00 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:56.669772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b3eba-9923-440e-83b9-02fb1c642a00 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:56.714835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a57da-e59c-49ea-b5aa-02967c01a230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:56.716039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a57da-e59c-49ea-b5aa-02967c01a230 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:56.716584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a57da-e59c-49ea-b5aa-02967c01a230 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:56.717059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a57da-e59c-49ea-b5aa-02967c01a230 admin admin] PATH_INFO: `/v3/projects/17c5623a13f9414e9ece79219ed9eece/users/313bfa4722374b62b973f6ea9ff549ac/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:56.749391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:56.752406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a57da-e59c-49ea-b5aa-02967c01a230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c5623a13f9414e9ece79219ed9eece, user_id=313bfa4722374b62b973f6ea9ff549ac, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:56.773867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a57da-e59c-49ea-b5aa-02967c01a230 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:56.883281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 878/1793] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:02:56 2022] PUT /identity/v3/projects/17c5623a13f9414e9ece79219ed9eece/users/313bfa4722374b62b973f6ea9ff549ac/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:56.885535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e24d8f-adc3-4698-b2eb-8e01c33482c1 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:56.886118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e24d8f-adc3-4698-b2eb-8e01c33482c1 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:56.886602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e24d8f-adc3-4698-b2eb-8e01c33482c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:56.891251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a57da-e59c-49ea-b5aa-02967c01a230 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:56.978250 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c8e24d8f-adc3-4698-b2eb-8e01c33482c1 None None] MFA Rules not processed for user `313bfa4722374b62b973f6ea9ff549ac`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:56.980673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c8e24d8f-adc3-4698-b2eb-8e01c33482c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:57.073504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a57da-e59c-49ea-b5aa-02967c01a230 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:57.093840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ed646-8ee2-40c8-9697-4c3ad426534b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:57.094909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ed646-8ee2-40c8-9697-4c3ad426534b service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:57.095550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ed646-8ee2-40c8-9697-4c3ad426534b service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:57.096010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ed646-8ee2-40c8-9697-4c3ad426534b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:57.139700 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d04ed646-8ee2-40c8-9697-4c3ad426534b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:57.243689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 916/1794] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:02:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:02:57.246154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e24d8f-adc3-4698-b2eb-8e01c33482c1 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:57.250171 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c8e24d8f-adc3-4698-b2eb-8e01c33482c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:57.268213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ed646-8ee2-40c8-9697-4c3ad426534b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:57.269738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ed646-8ee2-40c8-9697-4c3ad426534b service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:57.280898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 879/1795] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:02:57.334773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ed646-8ee2-40c8-9697-4c3ad426534b service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:57.349711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ed646-8ee2-40c8-9697-4c3ad426534b service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:57.355754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea3e7f-e574-4573-aeb3-ecb714fa5242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:57.356796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea3e7f-e574-4573-aeb3-ecb714fa5242 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:57.357280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea3e7f-e574-4573-aeb3-ecb714fa5242 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:57.357736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea3e7f-e574-4573-aeb3-ecb714fa5242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:57.359292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e24d8f-adc3-4698-b2eb-8e01c33482c1 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:57.361636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5aea3e7f-e574-4573-aeb3-ecb714fa5242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:57.369173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13789770-dae1-4595-8488-e702646399df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:57.370428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13789770-dae1-4595-8488-e702646399df admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:57.371048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13789770-dae1-4595-8488-e702646399df admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:57.371499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13789770-dae1-4595-8488-e702646399df admin admin] PATH_INFO: `/v3/projects/855a8b5a9d45448486790dc490720037` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:57.373507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13789770-dae1-4595-8488-e702646399df admin admin] RBAC: Authorizing `identity:delete_project(project_id=855a8b5a9d45448486790dc490720037)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:57.376544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13789770-dae1-4595-8488-e702646399df admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:57.486233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea3e7f-e574-4573-aeb3-ecb714fa5242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:57.488789 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea3e7f-e574-4573-aeb3-ecb714fa5242 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:57.507543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 880/1796] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:57.558523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-13789770-dae1-4595-8488-e702646399df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '855a8b5a9d45448486790dc490720037', 'request_id': 'req-13789770-dae1-4595-8488-e702646399df'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:57.569789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-13789770-dae1-4595-8488-e702646399df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '855a8b5a9d45448486790dc490720037', 'request_id': 'req-13789770-dae1-4595-8488-e702646399df'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:57.736847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-13789770-dae1-4595-8488-e702646399df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '855a8b5a9d45448486790dc490720037', 'request_id': 'req-13789770-dae1-4595-8488-e702646399df'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:57.763702 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-13789770-dae1-4595-8488-e702646399df admin admin] The token cache is being invalidate because project 855a8b5a9d45448486790dc490720037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:57.764276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-13789770-dae1-4595-8488-e702646399df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:57.765449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-13789770-dae1-4595-8488-e702646399df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:57.768277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 917/1797] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:57 2022] DELETE /identity/v3/projects/855a8b5a9d45448486790dc490720037 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:57.773667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea3e7f-e574-4573-aeb3-ecb714fa5242 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:57.790585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea3e7f-e574-4573-aeb3-ecb714fa5242 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:57.877286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:57.878195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:57.878535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:57.878868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] PATH_INFO: `/v3/users/ee8c303d966844d49909b14d85093d1d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:57.880868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee8c303d966844d49909b14d85093d1d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:57.883097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:58.284981 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee8c303d966844d49909b14d85093d1d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:58.345926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee8c303d966844d49909b14d85093d1d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:58.348116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee8c303d966844d49909b14d85093d1d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:58.361610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13789770-dae1-4595-8488-e702646399df admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:58.365723 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-13789770-dae1-4595-8488-e702646399df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:58.474764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee8c303d966844d49909b14d85093d1d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:58.525781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b01decd8-8ac4-4887-9633-0d0c8528ea14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee8c303d966844d49909b14d85093d1d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:58.544337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 881/1798] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:02:57 2022] DELETE /identity/v3/users/ee8c303d966844d49909b14d85093d1d => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:02:58.544586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13789770-dae1-4595-8488-e702646399df admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:58.685850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8c78d-7f2d-47e6-8bcd-b42a611e83d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:58.686743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8c78d-7f2d-47e6-8bcd-b42a611e83d0 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:58.687078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8c78d-7f2d-47e6-8bcd-b42a611e83d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:58.687543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8c78d-7f2d-47e6-8bcd-b42a611e83d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:58.690083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fdb8c78d-7f2d-47e6-8bcd-b42a611e83d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:58.708481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:58.815247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8c78d-7f2d-47e6-8bcd-b42a611e83d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:58.819067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8c78d-7f2d-47e6-8bcd-b42a611e83d0 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:58.844289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 918/1799] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:02:58.900431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a99a7d-d168-4fe8-94f1-26766a488b49 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:58.901290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a99a7d-d168-4fe8-94f1-26766a488b49 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:58.901421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a99a7d-d168-4fe8-94f1-26766a488b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:58.999689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e4a99a7d-d168-4fe8-94f1-26766a488b49 None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:02:59.001089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e4a99a7d-d168-4fe8-94f1-26766a488b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:59.031955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8c78d-7f2d-47e6-8bcd-b42a611e83d0 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:59.031955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fdb8c78d-7f2d-47e6-8bcd-b42a611e83d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:59.031955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8c78d-7f2d-47e6-8bcd-b42a611e83d0 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:59.039647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168de80-f74a-4f9c-abcb-2a983765af19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:59.042282 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168de80-f74a-4f9c-abcb-2a983765af19 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:59.043362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168de80-f74a-4f9c-abcb-2a983765af19 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:59.044099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168de80-f74a-4f9c-abcb-2a983765af19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:59.048504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7168de80-f74a-4f9c-abcb-2a983765af19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:59.125296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 882/1800] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:02:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:02:59.258355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168de80-f74a-4f9c-abcb-2a983765af19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:59.262343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168de80-f74a-4f9c-abcb-2a983765af19 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:59.275249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 919/1801] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:02:59.437622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a99a7d-d168-4fe8-94f1-26766a488b49 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:59.439706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e4a99a7d-d168-4fe8-94f1-26766a488b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:59.557720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a99a7d-d168-4fe8-94f1-26766a488b49 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:59.568238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:02:59.568932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:02:59.569216 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:02:59.574535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] PATH_INFO: `/v3/projects/7947ad8bf168403c830f8490364e9004` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:02:59.574535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7947ad8bf168403c830f8490364e9004)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:02:59.574535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:02:59.657543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168de80-f74a-4f9c-abcb-2a983765af19 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:59.659844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7168de80-f74a-4f9c-abcb-2a983765af19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:02:59.823021 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:02:59.825411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7947ad8bf168403c830f8490364e9004', 'request_id': 'req-bba8a8a1-97cc-4682-a770-04d408283900'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:59.839662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7947ad8bf168403c830f8490364e9004', 'request_id': 'req-bba8a8a1-97cc-4682-a770-04d408283900'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:59.857925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7947ad8bf168403c830f8490364e9004', 'request_id': 'req-bba8a8a1-97cc-4682-a770-04d408283900'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:59.884531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168de80-f74a-4f9c-abcb-2a983765af19 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:02:59.979101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] The token cache is being invalidate because project 7947ad8bf168403c830f8490364e9004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:02:59.979976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:59.980257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:02:59.981679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 883/1802] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:02:59 2022] DELETE /identity/v3/projects/7947ad8bf168403c830f8490364e9004 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:02:59.985653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:02:59.989914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:00.098258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e901f3-0edc-4ee4-beda-debc94d8f897 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:00.100028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e901f3-0edc-4ee4-beda-debc94d8f897 service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:00.100578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e901f3-0edc-4ee4-beda-debc94d8f897 service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:00.102476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e901f3-0edc-4ee4-beda-debc94d8f897 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:00.105562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-37e901f3-0edc-4ee4-beda-debc94d8f897 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:00.121718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:00.140664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8a8a1-97cc-4682-a770-04d408283900 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:00.147013 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094f14e-57ac-4a83-a32e-2db37eec03a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:00.147969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094f14e-57ac-4a83-a32e-2db37eec03a1 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:00.150501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094f14e-57ac-4a83-a32e-2db37eec03a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:00.151069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094f14e-57ac-4a83-a32e-2db37eec03a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:00.153884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2094f14e-57ac-4a83-a32e-2db37eec03a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:00.280267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e901f3-0edc-4ee4-beda-debc94d8f897 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:00.280267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e901f3-0edc-4ee4-beda-debc94d8f897 service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:00.295523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 920/1803] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:02:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 12:03:00.299241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203a0d8-eed4-4340-9c48-73176b90ebfb None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:00.299506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203a0d8-eed4-4340-9c48-73176b90ebfb None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:00.299782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203a0d8-eed4-4340-9c48-73176b90ebfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:00.336559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094f14e-57ac-4a83-a32e-2db37eec03a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:00.339962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094f14e-57ac-4a83-a32e-2db37eec03a1 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:00.342673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1203a0d8-eed4-4340-9c48-73176b90ebfb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:00.344070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1203a0d8-eed4-4340-9c48-73176b90ebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:00.374409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 884/1804] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:02:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:00.473191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ffd4-d4d0-4cad-9c66-e4f4025b63b7 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:00.473495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ffd4-d4d0-4cad-9c66-e4f4025b63b7 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:00.473859 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ffd4-d4d0-4cad-9c66-e4f4025b63b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:00.535731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 921/1805] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:00.547445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b7d9ffd4-d4d0-4cad-9c66-e4f4025b63b7 None None] MFA Rules not processed for user `073e43c049224af29e9f6d426ea47e84`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:00.551336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7b065-a99f-43c1-837d-57e23fe3d258 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:00.551880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7b065-a99f-43c1-837d-57e23fe3d258 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:00.554103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7b065-a99f-43c1-837d-57e23fe3d258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:00.561291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b7d9ffd4-d4d0-4cad-9c66-e4f4025b63b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:00.657082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-8ad7b065-a99f-43c1-837d-57e23fe3d258 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:00.665355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8ad7b065-a99f-43c1-837d-57e23fe3d258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:00.731145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 922/1806] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:00.738053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7b065-a99f-43c1-837d-57e23fe3d258 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:00.764745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7b065-a99f-43c1-837d-57e23fe3d258 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:00.827331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1370f5f-57bb-45bd-9700-affec98645d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:00.831760 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1370f5f-57bb-45bd-9700-affec98645d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:00.832584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1370f5f-57bb-45bd-9700-affec98645d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:00.833136 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1370f5f-57bb-45bd-9700-affec98645d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:00.834915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1370f5f-57bb-45bd-9700-affec98645d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:00.835454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1370f5f-57bb-45bd-9700-affec98645d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:00.845502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1370f5f-57bb-45bd-9700-affec98645d7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:00.930995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 923/1807] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:03:00.940032 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1370f5f-57bb-45bd-9700-affec98645d7 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:00.957158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 885/1808] 10.222.0.43 () {58 vars in 1015 bytes} [Tue Oct 4 12:03:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:00.979692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1370f5f-57bb-45bd-9700-affec98645d7 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:01.007233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c77ab9-7149-4cf4-be24-1a7052e7e2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:01.008060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c77ab9-7149-4cf4-be24-1a7052e7e2ad admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:01.011713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c77ab9-7149-4cf4-be24-1a7052e7e2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:01.011713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c77ab9-7149-4cf4-be24-1a7052e7e2ad admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:01.011713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c77ab9-7149-4cf4-be24-1a7052e7e2ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:01.024814 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c77ab9-7149-4cf4-be24-1a7052e7e2ad admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:01.067471 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:01.095934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ffd4-d4d0-4cad-9c66-e4f4025b63b7 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:01.098313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b7d9ffd4-d4d0-4cad-9c66-e4f4025b63b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:01.167894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ffd4-d4d0-4cad-9c66-e4f4025b63b7 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:01.192636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f75b-c017-4d5e-a925-f6afd4eadba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:01.194741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f75b-c017-4d5e-a925-f6afd4eadba1 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:01.196469 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f75b-c017-4d5e-a925-f6afd4eadba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:01.198147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355f75b-c017-4d5e-a925-f6afd4eadba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:01.237473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355f75b-c017-4d5e-a925-f6afd4eadba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:01.243485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355f75b-c017-4d5e-a925-f6afd4eadba1 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:01.309859 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 886/1809] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:03:01.474977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 924/1810] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:00 2022] POST /identity/v3/projects => generated 381 bytes in 536 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:01.482011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f75b-c017-4d5e-a925-f6afd4eadba1 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:01.501775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355f75b-c017-4d5e-a925-f6afd4eadba1 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:01.607828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2743da-9f81-414c-aa70-7a94da5874ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:01.608643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2743da-9f81-414c-aa70-7a94da5874ec admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:01.608943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2743da-9f81-414c-aa70-7a94da5874ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:01.609374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2743da-9f81-414c-aa70-7a94da5874ec admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:01.610141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2743da-9f81-414c-aa70-7a94da5874ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:01.612391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2743da-9f81-414c-aa70-7a94da5874ec admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:01.705960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:01.790976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 887/1811] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:01 2022] POST /identity/v3/users => generated 291 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:03:01.797805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c77ab9-7149-4cf4-be24-1a7052e7e2ad admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:01.835889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c77ab9-7149-4cf4-be24-1a7052e7e2ad admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:01.848635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cbbff-54e3-4d78-a304-0b443095e47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:01.850222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cbbff-54e3-4d78-a304-0b443095e47b admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:01.852165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cbbff-54e3-4d78-a304-0b443095e47b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:01.854999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cbbff-54e3-4d78-a304-0b443095e47b admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:01.854999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cbbff-54e3-4d78-a304-0b443095e47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:01.858281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cbbff-54e3-4d78-a304-0b443095e47b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:01.886789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 925/1812] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:01.890655 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2743da-9f81-414c-aa70-7a94da5874ec admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:01.929168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2743da-9f81-414c-aa70-7a94da5874ec admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:01.939370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06661841-7dd3-4855-a282-b252c297b5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:01.940895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06661841-7dd3-4855-a282-b252c297b5dc admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:01.941371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06661841-7dd3-4855-a282-b252c297b5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:01.942763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06661841-7dd3-4855-a282-b252c297b5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:01.943664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06661841-7dd3-4855-a282-b252c297b5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:01.948608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06661841-7dd3-4855-a282-b252c297b5dc admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:01.985996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cbbff-54e3-4d78-a304-0b443095e47b admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:02.004328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 888/1813] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:02.012802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cbbff-54e3-4d78-a304-0b443095e47b admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:02.032016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf802d3-582d-4bd0-8a6c-5ac97ee4d1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:02.035450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf802d3-582d-4bd0-8a6c-5ac97ee4d1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:02.037905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:02.038659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf802d3-582d-4bd0-8a6c-5ac97ee4d1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:02.040748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf802d3-582d-4bd0-8a6c-5ac97ee4d1bc admin admin] PATH_INFO: `/v3/projects/94880526dacb4078ba7b6adf1605c05a/users/ae4c8ec0b9c746d89ffdfbaa92351e84/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:02.120845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf802d3-582d-4bd0-8a6c-5ac97ee4d1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=94880526dacb4078ba7b6adf1605c05a, user_id=ae4c8ec0b9c746d89ffdfbaa92351e84, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:02.136501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf802d3-582d-4bd0-8a6c-5ac97ee4d1bc admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:02.466264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 926/1814] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:01 2022] PUT /identity/v3/projects/94880526dacb4078ba7b6adf1605c05a/users/ae4c8ec0b9c746d89ffdfbaa92351e84/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:02.503362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b182a1-a07a-4a4c-b4a9-fe159117c7e8 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:02.505985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b182a1-a07a-4a4c-b4a9-fe159117c7e8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:02.506763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b182a1-a07a-4a4c-b4a9-fe159117c7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:02.567994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-35b182a1-a07a-4a4c-b4a9-fe159117c7e8 None None] MFA Rules not processed for user `ae4c8ec0b9c746d89ffdfbaa92351e84`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:02.567994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-35b182a1-a07a-4a4c-b4a9-fe159117c7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:02.883379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 889/1815] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:02.890478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf802d3-582d-4bd0-8a6c-5ac97ee4d1bc admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:03.134050 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:03.174580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf802d3-582d-4bd0-8a6c-5ac97ee4d1bc admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:03.185764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fd7ff-2e48-4f78-bfd4-ffc44f5be670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:03.187368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fd7ff-2e48-4f78-bfd4-ffc44f5be670 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:03.189202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fd7ff-2e48-4f78-bfd4-ffc44f5be670 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:03.190060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fd7ff-2e48-4f78-bfd4-ffc44f5be670 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:03.191552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fd7ff-2e48-4f78-bfd4-ffc44f5be670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:03.197546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fd7ff-2e48-4f78-bfd4-ffc44f5be670 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:03.387531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 927/1816] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:02 2022] POST /identity/v3/projects => generated 379 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:03.393139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b182a1-a07a-4a4c-b4a9-fe159117c7e8 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:03.428098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b182a1-a07a-4a4c-b4a9-fe159117c7e8 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:03.510473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:03.638288 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b88e18-2d3f-4f38-ac18-2eb2d2ae3e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:03.640815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b88e18-2d3f-4f38-ac18-2eb2d2ae3e60 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:03.642232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b88e18-2d3f-4f38-ac18-2eb2d2ae3e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:03.643818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b88e18-2d3f-4f38-ac18-2eb2d2ae3e60 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:03.646245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b88e18-2d3f-4f38-ac18-2eb2d2ae3e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:03.668443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b88e18-2d3f-4f38-ac18-2eb2d2ae3e60 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:03.848140 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 890/1817] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:03 2022] POST /identity/v3/users => generated 296 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:03.853059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fd7ff-2e48-4f78-bfd4-ffc44f5be670 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:03.877794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fd7ff-2e48-4f78-bfd4-ffc44f5be670 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:03.916420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1fcdb-06a4-4c4f-b0ee-f3cd83312d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:03.917529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1fcdb-06a4-4c4f-b0ee-f3cd83312d3b admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:03.917989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1fcdb-06a4-4c4f-b0ee-f3cd83312d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:03.918462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1fcdb-06a4-4c4f-b0ee-f3cd83312d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:03.919691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1fcdb-06a4-4c4f-b0ee-f3cd83312d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:03.923026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1fcdb-06a4-4c4f-b0ee-f3cd83312d3b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:04.003446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 928/1818] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:03:04.009992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b88e18-2d3f-4f38-ac18-2eb2d2ae3e60 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:04.035431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b88e18-2d3f-4f38-ac18-2eb2d2ae3e60 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:04.069769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7de36-01c8-4171-a8fd-15f1df1bcd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:04.072249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7de36-01c8-4171-a8fd-15f1df1bcd5f admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:04.073401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7de36-01c8-4171-a8fd-15f1df1bcd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:04.075702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7de36-01c8-4171-a8fd-15f1df1bcd5f admin admin] PATH_INFO: `/v3/projects/e0ccf73aac884cf29bf66dc92f90768a/users/1a3ec36bce7a476985647979f8ce2807/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:04.117359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7de36-01c8-4171-a8fd-15f1df1bcd5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ccf73aac884cf29bf66dc92f90768a, user_id=1a3ec36bce7a476985647979f8ce2807, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:04.123819 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7de36-01c8-4171-a8fd-15f1df1bcd5f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:04.343997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:04.389590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 891/1819] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:04 2022] PUT /identity/v3/projects/e0ccf73aac884cf29bf66dc92f90768a/users/1a3ec36bce7a476985647979f8ce2807/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:04.390028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1fcdb-06a4-4c4f-b0ee-f3cd83312d3b admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:04.415397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1fcdb-06a4-4c4f-b0ee-f3cd83312d3b admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:04.587675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f91552-4155-4a3e-8a5d-260222ba263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:04.589006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f91552-4155-4a3e-8a5d-260222ba263f admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:04.595477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f91552-4155-4a3e-8a5d-260222ba263f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:04.595477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f91552-4155-4a3e-8a5d-260222ba263f admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:04.595477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f91552-4155-4a3e-8a5d-260222ba263f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:04.602287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f91552-4155-4a3e-8a5d-260222ba263f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:04.645307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 929/1820] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:04.649290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7de36-01c8-4171-a8fd-15f1df1bcd5f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:04.666094 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7de36-01c8-4171-a8fd-15f1df1bcd5f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:04.691724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefc97d-6352-49c6-b687-bcff0a8936d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:04.693130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefc97d-6352-49c6-b687-bcff0a8936d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:04.693405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefc97d-6352-49c6-b687-bcff0a8936d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:04.693861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefc97d-6352-49c6-b687-bcff0a8936d8 admin admin] PATH_INFO: `/v3/projects/e0ccf73aac884cf29bf66dc92f90768a/users/1a3ec36bce7a476985647979f8ce2807/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:04.716208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefc97d-6352-49c6-b687-bcff0a8936d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ccf73aac884cf29bf66dc92f90768a, user_id=1a3ec36bce7a476985647979f8ce2807, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:04.719344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefc97d-6352-49c6-b687-bcff0a8936d8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:04.757829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 892/1821] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:04 2022] PUT /identity/v3/projects/e0ccf73aac884cf29bf66dc92f90768a/users/1a3ec36bce7a476985647979f8ce2807/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:04.761696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e3cc2-0f34-40bf-add7-e2a3dd27c308 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:04.762260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e3cc2-0f34-40bf-add7-e2a3dd27c308 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:04.762695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e3cc2-0f34-40bf-add7-e2a3dd27c308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:04.819937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-064e3cc2-0f34-40bf-add7-e2a3dd27c308 None None] MFA Rules not processed for user `1a3ec36bce7a476985647979f8ce2807`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:04.822496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-064e3cc2-0f34-40bf-add7-e2a3dd27c308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:04.994983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:05.091396 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:05.170671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 930/1822] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:05.523388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefc97d-6352-49c6-b687-bcff0a8936d8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:05.877433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefc97d-6352-49c6-b687-bcff0a8936d8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:05.899091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547fc2b-2066-49a5-ace1-3ef134f95e61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:05.904270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547fc2b-2066-49a5-ace1-3ef134f95e61 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:05.905228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547fc2b-2066-49a5-ace1-3ef134f95e61 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:05.906318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547fc2b-2066-49a5-ace1-3ef134f95e61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:05.962481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547fc2b-2066-49a5-ace1-3ef134f95e61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:05.971702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547fc2b-2066-49a5-ace1-3ef134f95e61 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:05.987632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8aa4f-c272-4ce3-b38e-074dbaa7bb57 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:05.988633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8aa4f-c272-4ce3-b38e-074dbaa7bb57 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:05.988633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8aa4f-c272-4ce3-b38e-074dbaa7bb57 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:05.990239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 931/1823] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:03:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:03:06.053465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 893/1824] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:06.077798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432cb6b-8aee-48e6-8cf8-1a2f652afa43 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:06.078326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432cb6b-8aee-48e6-8cf8-1a2f652afa43 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:06.078914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432cb6b-8aee-48e6-8cf8-1a2f652afa43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:06.140564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-3432cb6b-8aee-48e6-8cf8-1a2f652afa43 None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:06.146249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3432cb6b-8aee-48e6-8cf8-1a2f652afa43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:06.389023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 932/1825] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:03:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:06.604091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547fc2b-2066-49a5-ace1-3ef134f95e61 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:06.606354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f547fc2b-2066-49a5-ace1-3ef134f95e61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:06.630132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547fc2b-2066-49a5-ace1-3ef134f95e61 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:06.651946 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2ecaa-3816-4f32-8abd-a7a74b14449c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:06.653037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2ecaa-3816-4f32-8abd-a7a74b14449c service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:06.653887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2ecaa-3816-4f32-8abd-a7a74b14449c service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:06.654407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2ecaa-3816-4f32-8abd-a7a74b14449c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:06.681863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2ecaa-3816-4f32-8abd-a7a74b14449c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:06.684202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2ecaa-3816-4f32-8abd-a7a74b14449c service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:06.717726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 894/1826] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:06.744786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:06.931759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:08.485523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad29f-0598-46fd-ba4d-326e844b5f56 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:08.486601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad29f-0598-46fd-ba4d-326e844b5f56 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:08.486777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad29f-0598-46fd-ba4d-326e844b5f56 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:08.492696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 933/1827] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:03:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:03:08.515203 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469647b-3c80-4396-90d3-c9a5541031d5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:08.516053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469647b-3c80-4396-90d3-c9a5541031d5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:08.516053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469647b-3c80-4396-90d3-c9a5541031d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:08.594926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-4469647b-3c80-4396-90d3-c9a5541031d5 None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:08.596257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4469647b-3c80-4396-90d3-c9a5541031d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:08.636804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 895/1828] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:03:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:08.663192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ad29f-0598-46fd-ba4d-326e844b5f56 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:08.667848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-be5ad29f-0598-46fd-ba4d-326e844b5f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:08.746286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ad29f-0598-46fd-ba4d-326e844b5f56 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:08.759478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bad87-37d0-433f-8a26-04cce9a11cf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:08.761374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bad87-37d0-433f-8a26-04cce9a11cf8 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:08.762275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bad87-37d0-433f-8a26-04cce9a11cf8 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:08.765260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bad87-37d0-433f-8a26-04cce9a11cf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:08.804956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bad87-37d0-433f-8a26-04cce9a11cf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:08.812733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bad87-37d0-433f-8a26-04cce9a11cf8 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:08.859163 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 934/1829] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:08.977812 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:09.233469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:09.750883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4469647b-3c80-4396-90d3-c9a5541031d5 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:09.785195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4469647b-3c80-4396-90d3-c9a5541031d5 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:09.805934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c207-4e13-44d4-9685-4d9aa30b8b9e tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a3ec36bce7a476985647979f8ce2807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0ccf73aac884cf29bf66dc92f90768a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:09.807513 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c207-4e13-44d4-9685-4d9aa30b8b9e tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:09.809637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c207-4e13-44d4-9685-4d9aa30b8b9e tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:09.810438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c207-4e13-44d4-9685-4d9aa30b8b9e tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:09.815134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256c207-4e13-44d4-9685-4d9aa30b8b9e tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:09.816161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256c207-4e13-44d4-9685-4d9aa30b8b9e tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:09.822168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256c207-4e13-44d4-9685-4d9aa30b8b9e tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:09.869729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 896/1830] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:03:09.876912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bad87-37d0-433f-8a26-04cce9a11cf8 service nova] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:09.902843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bad87-37d0-433f-8a26-04cce9a11cf8 service nova] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:09.908636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52d1d5-3cd5-4023-a1ff-6dd9778c6cb2 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a3ec36bce7a476985647979f8ce2807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0ccf73aac884cf29bf66dc92f90768a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:09.909531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52d1d5-3cd5-4023-a1ff-6dd9778c6cb2 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:09.909907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52d1d5-3cd5-4023-a1ff-6dd9778c6cb2 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:09.910298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52d1d5-3cd5-4023-a1ff-6dd9778c6cb2 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:09.911854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52d1d5-3cd5-4023-a1ff-6dd9778c6cb2 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:09.914172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52d1d5-3cd5-4023-a1ff-6dd9778c6cb2 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:10.064441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 935/1831] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:09 2022] POST /identity/v3/projects => generated 355 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:03:10.069523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c207-4e13-44d4-9685-4d9aa30b8b9e tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:10.088245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c207-4e13-44d4-9685-4d9aa30b8b9e tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:10.144287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52d1d5-3cd5-4023-a1ff-6dd9778c6cb2 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:10.146789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cd52d1d5-3cd5-4023-a1ff-6dd9778c6cb2 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:10.202615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543eeef-262e-4d04-b323-d78e715fd8eb tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a3ec36bce7a476985647979f8ce2807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0ccf73aac884cf29bf66dc92f90768a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:10.203561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543eeef-262e-4d04-b323-d78e715fd8eb tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:10.204014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543eeef-262e-4d04-b323-d78e715fd8eb tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:10.204506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543eeef-262e-4d04-b323-d78e715fd8eb tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:10.205359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543eeef-262e-4d04-b323-d78e715fd8eb tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:10.209498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543eeef-262e-4d04-b323-d78e715fd8eb tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:10.211653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543eeef-262e-4d04-b323-d78e715fd8eb tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:10.251877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 897/1832] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:03:10.295472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52d1d5-3cd5-4023-a1ff-6dd9778c6cb2 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:10.304067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:10.305207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:10.305659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:10.306107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] PATH_INFO: `/v3/users/c1592d75faf54feebedca83d35e3b971` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:10.307652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1592d75faf54feebedca83d35e3b971)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:10.311079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:10.562655 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543eeef-262e-4d04-b323-d78e715fd8eb tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:10.583887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543eeef-262e-4d04-b323-d78e715fd8eb tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:10.602102 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1592d75faf54feebedca83d35e3b971'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:10.603886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1592d75faf54feebedca83d35e3b971'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:10.625448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a3ec36bce7a476985647979f8ce2807', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0ccf73aac884cf29bf66dc92f90768a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:10.626026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1592d75faf54feebedca83d35e3b971'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:10.636342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1592d75faf54feebedca83d35e3b971'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:10.640488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:10.641989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:10.642392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] PATH_INFO: `/v3/projects/4db7069c112944cabd4026fba99e2beb` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:10.645844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4db7069c112944cabd4026fba99e2beb)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:10.654450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:10.753060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8caaf0aa-94e7-44ea-9c5b-7fae40f5ba50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1592d75faf54feebedca83d35e3b971'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:10.803095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 936/1833] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:10 2022] DELETE /identity/v3/users/c1592d75faf54feebedca83d35e3b971 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:10.808017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b7779-5159-40b6-941a-c08865981e97 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:10.808644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b7779-5159-40b6-941a-c08865981e97 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:10.808978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b7779-5159-40b6-941a-c08865981e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:10.859074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-6a8b7779-5159-40b6-941a-c08865981e97 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:10.860866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6a8b7779-5159-40b6-941a-c08865981e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:10.875145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db7069c112944cabd4026fba99e2beb', 'request_id': 'req-75001916-77d7-48bd-91f3-f3d0a1af8467'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:10.906189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db7069c112944cabd4026fba99e2beb', 'request_id': 'req-75001916-77d7-48bd-91f3-f3d0a1af8467'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:10.926589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db7069c112944cabd4026fba99e2beb', 'request_id': 'req-75001916-77d7-48bd-91f3-f3d0a1af8467'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:11.052484 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 937/1834] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:11.070104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b7779-5159-40b6-941a-c08865981e97 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:11.072584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6a8b7779-5159-40b6-941a-c08865981e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:11.085115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] The token cache is being invalidate because project 4db7069c112944cabd4026fba99e2beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:11.086144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:11.088044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:11.093663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 898/1835] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:10 2022] DELETE /identity/v3/projects/4db7069c112944cabd4026fba99e2beb => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:11.182425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:11.184425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:11.208579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75001916-77d7-48bd-91f3-f3d0a1af8467 tempest-VolumeQuotaClassesTest-241204430 tempest-VolumeQuotaClassesTest-241204430-project-admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:11.211131 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b7779-5159-40b6-941a-c08865981e97 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:11.217883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aac38e-63ab-42da-a380-18b7195baa48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:11.218863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aac38e-63ab-42da-a380-18b7195baa48 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:11.219340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aac38e-63ab-42da-a380-18b7195baa48 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:11.220157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aac38e-63ab-42da-a380-18b7195baa48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:11.224320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-10aac38e-63ab-42da-a380-18b7195baa48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:11.228910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:11.234240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:11.235226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:11.235962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] PATH_INFO: `/v3/users/c6dcf85e2ed14685bb62de7a6f47a0c4` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:11.243692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6dcf85e2ed14685bb62de7a6f47a0c4)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:11.249717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:11.253690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aac38e-63ab-42da-a380-18b7195baa48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:11.257155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aac38e-63ab-42da-a380-18b7195baa48 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:11.288910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 938/1836] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:11.404961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:11.601690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:11.614161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6dcf85e2ed14685bb62de7a6f47a0c4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:11.657088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6dcf85e2ed14685bb62de7a6f47a0c4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:11.660839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6dcf85e2ed14685bb62de7a6f47a0c4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:11.892565 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6dcf85e2ed14685bb62de7a6f47a0c4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:11.954830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6dcf85e2ed14685bb62de7a6f47a0c4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:11.980502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 899/1837] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:11 2022] DELETE /identity/v3/users/c6dcf85e2ed14685bb62de7a6f47a0c4 => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:11.987590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25381b-c6dd-4fa6-9af8-416802b74178 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:11.988146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25381b-c6dd-4fa6-9af8-416802b74178 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:11.990967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25381b-c6dd-4fa6-9af8-416802b74178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:12.025700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9c25381b-c6dd-4fa6-9af8-416802b74178 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:12.028130 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9c25381b-c6dd-4fa6-9af8-416802b74178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:12.114024 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:12.117616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:12.396681 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 939/1838] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:12.560344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b1f41-902f-4c5a-b743-81163063f115 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:12.574984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:12.574984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:12.574984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:12.574984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] PATH_INFO: `/v3/projects/5e83563c63f44551bca1c6debe26340e` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:12.576368 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e83563c63f44551bca1c6debe26340e)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:12.580428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:12.651901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25381b-c6dd-4fa6-9af8-416802b74178 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:12.654968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9c25381b-c6dd-4fa6-9af8-416802b74178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:12.717586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25381b-c6dd-4fa6-9af8-416802b74178 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:12.742804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:12.746309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:12.750352 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:12.751427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] PATH_INFO: `/v3/users/ae4c8ec0b9c746d89ffdfbaa92351e84` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:12.755817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae4c8ec0b9c746d89ffdfbaa92351e84)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:12.759980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:12.893786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e83563c63f44551bca1c6debe26340e', 'request_id': 'req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:12.923759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e83563c63f44551bca1c6debe26340e', 'request_id': 'req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:12.967600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e83563c63f44551bca1c6debe26340e', 'request_id': 'req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.078631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae4c8ec0b9c746d89ffdfbaa92351e84'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.081888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae4c8ec0b9c746d89ffdfbaa92351e84'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.108729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae4c8ec0b9c746d89ffdfbaa92351e84'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.114167 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae4c8ec0b9c746d89ffdfbaa92351e84'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.134037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] The token cache is being invalidate because project 5e83563c63f44551bca1c6debe26340e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:13.134653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.138316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.145783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 900/1839] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:12 2022] DELETE /identity/v3/projects/5e83563c63f44551bca1c6debe26340e => generated 0 bytes in 1034 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:13.148024 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:13.155909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:13.271583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae4c8ec0b9c746d89ffdfbaa92351e84'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.371068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 940/1840] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:12 2022] DELETE /identity/v3/users/ae4c8ec0b9c746d89ffdfbaa92351e84 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:13.373516 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:13.379604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:13.464569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7093ac-af19-4adf-9e1f-f73e844d1fc7 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:13.639113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a09c99-460c-4f9b-a812-24e07e9b72eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:13.640793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a09c99-460c-4f9b-a812-24e07e9b72eb service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:13.641682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a09c99-460c-4f9b-a812-24e07e9b72eb service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:13.642674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a09c99-460c-4f9b-a812-24e07e9b72eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:13.647630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-20a09c99-460c-4f9b-a812-24e07e9b72eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:13.664609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29ca38-b2d3-4438-b2fc-cd5906d975e8 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:13.673690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:13.674817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:13.675287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:13.676050 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] PATH_INFO: `/v3/users/9585a44aa90b4218ae69fd4700aeafca` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:13.678855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9585a44aa90b4218ae69fd4700aeafca)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:13.681530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a09c99-460c-4f9b-a812-24e07e9b72eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:13.685489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:13.685885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a09c99-460c-4f9b-a812-24e07e9b72eb service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:13.725742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 901/1841] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:13.728780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a09c99-460c-4f9b-a812-24e07e9b72eb service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:13.755483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-20a09c99-460c-4f9b-a812-24e07e9b72eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:13.775335 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a09c99-460c-4f9b-a812-24e07e9b72eb service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:13.793446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:13.801897 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:13.803298 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:13.804158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] PATH_INFO: `/v3/users/d727e4a4e9f045d3a7eca8e3a3a5be2a` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:13.808073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d727e4a4e9f045d3a7eca8e3a3a5be2a)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:13.810678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:13.981908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9585a44aa90b4218ae69fd4700aeafca'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.982711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9585a44aa90b4218ae69fd4700aeafca'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.997103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9585a44aa90b4218ae69fd4700aeafca'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:13.998148 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9585a44aa90b4218ae69fd4700aeafca'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:14.136887 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-bf4c02dd-2f05-4db1-a72e-be2ff17d050a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9585a44aa90b4218ae69fd4700aeafca'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:14.199858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 941/1842] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:13 2022] DELETE /identity/v3/users/9585a44aa90b4218ae69fd4700aeafca => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:14.211252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04083778-441d-434f-8a0d-bacd6b69da7b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:14.212822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04083778-441d-434f-8a0d-bacd6b69da7b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:14.213604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04083778-441d-434f-8a0d-bacd6b69da7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:14.231073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd727e4a4e9f045d3a7eca8e3a3a5be2a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:14.250324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd727e4a4e9f045d3a7eca8e3a3a5be2a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:14.252553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd727e4a4e9f045d3a7eca8e3a3a5be2a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:14.261217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-04083778-441d-434f-8a0d-bacd6b69da7b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:14.263452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-04083778-441d-434f-8a0d-bacd6b69da7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:14.285926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:14.367516 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 942/1843] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:14.393226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd727e4a4e9f045d3a7eca8e3a3a5be2a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:14.415171 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04083778-441d-434f-8a0d-bacd6b69da7b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:14.416711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-04083778-441d-434f-8a0d-bacd6b69da7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:14.423347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-18d42f06-4466-4453-90fc-fbac03689958 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd727e4a4e9f045d3a7eca8e3a3a5be2a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:14.453213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 902/1844] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:13 2022] DELETE /identity/v3/users/d727e4a4e9f045d3a7eca8e3a3a5be2a => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:14.459543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b536bfb-3994-44cc-ab47-5b74bd14ac09 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:14.459837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b536bfb-3994-44cc-ab47-5b74bd14ac09 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:14.460093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b536bfb-3994-44cc-ab47-5b74bd14ac09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:14.536204 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7b536bfb-3994-44cc-ab47-5b74bd14ac09 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:14.541778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7b536bfb-3994-44cc-ab47-5b74bd14ac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:14.601890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04083778-441d-434f-8a0d-bacd6b69da7b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:14.693434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e18551-20a7-4346-a10b-4c6a64119d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:14.694481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e18551-20a7-4346-a10b-4c6a64119d09 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:14.694481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e18551-20a7-4346-a10b-4c6a64119d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:14.695000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e18551-20a7-4346-a10b-4c6a64119d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:14.850397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e18551-20a7-4346-a10b-4c6a64119d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:14.852315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e18551-20a7-4346-a10b-4c6a64119d09 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:14.867166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:14.868952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 943/1845] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:03:14.896294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 903/1846] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:14.978067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e18551-20a7-4346-a10b-4c6a64119d09 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:14.980509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-16e18551-20a7-4346-a10b-4c6a64119d09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:15.115882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e18551-20a7-4346-a10b-4c6a64119d09 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:15.115882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176806f0-b6ef-4356-b540-f3ac3f4650bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:15.115882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176806f0-b6ef-4356-b540-f3ac3f4650bf service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:15.115882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176806f0-b6ef-4356-b540-f3ac3f4650bf service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:15.117440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176806f0-b6ef-4356-b540-f3ac3f4650bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:15.117440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176806f0-b6ef-4356-b540-f3ac3f4650bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:15.117440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176806f0-b6ef-4356-b540-f3ac3f4650bf service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:15.117440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 944/1847] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:15.980014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b536bfb-3994-44cc-ab47-5b74bd14ac09 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:15.983615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7b536bfb-3994-44cc-ab47-5b74bd14ac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:16.015706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b536bfb-3994-44cc-ab47-5b74bd14ac09 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:16.021261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:16.022362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:16.022864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:16.023605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] PATH_INFO: `/v3/projects/5993fd3d54354600aca8bf742f9be2ed` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:16.027110 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5993fd3d54354600aca8bf742f9be2ed)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:16.030845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:16.134301 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176806f0-b6ef-4356-b540-f3ac3f4650bf service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:16.136304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-176806f0-b6ef-4356-b540-f3ac3f4650bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:16.173714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176806f0-b6ef-4356-b540-f3ac3f4650bf service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:16.192212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:16.199054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:16.199559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:16.214866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] PATH_INFO: `/v3/projects/94880526dacb4078ba7b6adf1605c05a` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:16.218490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94880526dacb4078ba7b6adf1605c05a)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:16.221717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:16.250051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5993fd3d54354600aca8bf742f9be2ed', 'request_id': 'req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.258418 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5993fd3d54354600aca8bf742f9be2ed', 'request_id': 'req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.275296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5993fd3d54354600aca8bf742f9be2ed', 'request_id': 'req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.418622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] The token cache is being invalidate because project 5993fd3d54354600aca8bf742f9be2ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:16.419515 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.421043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.424587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 904/1848] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:15 2022] DELETE /identity/v3/projects/5993fd3d54354600aca8bf742f9be2ed => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:16.426689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:16.428877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:16.459012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94880526dacb4078ba7b6adf1605c05a', 'request_id': 'req-eb22cf30-ebd0-4566-a438-17032c6cbd29'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.473609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94880526dacb4078ba7b6adf1605c05a', 'request_id': 'req-eb22cf30-ebd0-4566-a438-17032c6cbd29'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.549065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52e9f6-24ae-4cd9-9158-1b7c003d0945 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:16.582638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94880526dacb4078ba7b6adf1605c05a', 'request_id': 'req-eb22cf30-ebd0-4566-a438-17032c6cbd29'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.603830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] The token cache is being invalidate because project 94880526dacb4078ba7b6adf1605c05a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:16.604155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.605003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.606561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 945/1849] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:16 2022] DELETE /identity/v3/projects/94880526dacb4078ba7b6adf1605c05a => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:16.607626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:16.610656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:16.661934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:16.663805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:16.663805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:16.663805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] PATH_INFO: `/v3/projects/242b49781f10432c879829f777cfe789` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:16.666020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242b49781f10432c879829f777cfe789)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:16.674010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:16.711143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22cf30-ebd0-4566-a438-17032c6cbd29 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:16.716058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:16.716755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:16.717034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:16.717327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] PATH_INFO: `/v3/projects/3c5b07104b3e4d96b2391cbda2e72a32` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:16.718566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c5b07104b3e4d96b2391cbda2e72a32)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:16.720549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:16.807397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242b49781f10432c879829f777cfe789', 'request_id': 'req-219264cd-591e-4319-b595-49967dd19181'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.813466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242b49781f10432c879829f777cfe789', 'request_id': 'req-219264cd-591e-4319-b595-49967dd19181'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.825332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242b49781f10432c879829f777cfe789', 'request_id': 'req-219264cd-591e-4319-b595-49967dd19181'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.845525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c5b07104b3e4d96b2391cbda2e72a32', 'request_id': 'req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.856139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c5b07104b3e4d96b2391cbda2e72a32', 'request_id': 'req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.955718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] The token cache is being invalidate because project 242b49781f10432c879829f777cfe789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:16.955718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.955718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:16.955718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 905/1850] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:16 2022] DELETE /identity/v3/projects/242b49781f10432c879829f777cfe789 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:16.956157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:16.958869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:16.977737 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:17.036887 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c5b07104b3e4d96b2391cbda2e72a32', 'request_id': 'req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.054161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219264cd-591e-4319-b595-49967dd19181 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:17.059762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] The token cache is being invalidate because project 3c5b07104b3e4d96b2391cbda2e72a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:17.060088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.060825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.068218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 946/1851] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:16 2022] DELETE /identity/v3/projects/3c5b07104b3e4d96b2391cbda2e72a32 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:17.069278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:17.070910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:17.146573 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:17.147616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:17.147972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:17.148498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] PATH_INFO: `/v3/users/65339b009fe147f99df52b13c024afab` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:17.149988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65339b009fe147f99df52b13c024afab)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:17.156353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:17.169696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5c21b-ab53-428b-8a5d-7dae6b1e9ff2 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:17.186225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:17.186894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:17.187176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:17.188810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] PATH_INFO: `/v3/users/1a3ec36bce7a476985647979f8ce2807` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:17.189811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a3ec36bce7a476985647979f8ce2807)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:17.202210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:17.544011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65339b009fe147f99df52b13c024afab'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.545622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a3ec36bce7a476985647979f8ce2807'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.546730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a3ec36bce7a476985647979f8ce2807'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.558114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65339b009fe147f99df52b13c024afab'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.559052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a3ec36bce7a476985647979f8ce2807'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.559836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65339b009fe147f99df52b13c024afab'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.561026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a3ec36bce7a476985647979f8ce2807'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.638274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:17.752437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3904d76-7ea1-426f-8e18-d87cdcc51872 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a3ec36bce7a476985647979f8ce2807'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.779894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 947/1852] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:17 2022] DELETE /identity/v3/users/1a3ec36bce7a476985647979f8ce2807 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:17.782065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b965128-8dbe-4e6f-8f78-ea7cb5f8c391 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:17.782512 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b965128-8dbe-4e6f-8f78-ea7cb5f8c391 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:17.782952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b965128-8dbe-4e6f-8f78-ea7cb5f8c391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:17.814126 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-7b965128-8dbe-4e6f-8f78-ea7cb5f8c391 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:17.820226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7b965128-8dbe-4e6f-8f78-ea7cb5f8c391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:17.890036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65339b009fe147f99df52b13c024afab'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.909639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65339b009fe147f99df52b13c024afab'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:17.917921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 906/1853] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:16 2022] DELETE /identity/v3/users/65339b009fe147f99df52b13c024afab => generated 0 bytes in 963 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:17.918855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:17.919679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 948/1854] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:17.928425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:17.929095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6f21c-9c5c-4d8a-b793-62b7ff89fbb9 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:17.929473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6f21c-9c5c-4d8a-b793-62b7ff89fbb9 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:17.929758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6f21c-9c5c-4d8a-b793-62b7ff89fbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:17.978165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-3ef6f21c-9c5c-4d8a-b793-62b7ff89fbb9 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:17.981488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3ef6f21c-9c5c-4d8a-b793-62b7ff89fbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:18.021881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758b3ee-e26a-4ef4-a835-417c2ae05173 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:18.028050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:18.029014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:18.029222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:18.029721 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] PATH_INFO: `/v3/users/3734d8a1452f4e28a48e747f46d98b29` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:18.031227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3734d8a1452f4e28a48e747f46d98b29)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:18.033457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:18.107831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 949/1855] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:18.107831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6f21c-9c5c-4d8a-b793-62b7ff89fbb9 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:18.107831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3ef6f21c-9c5c-4d8a-b793-62b7ff89fbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:18.121088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6f21c-9c5c-4d8a-b793-62b7ff89fbb9 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:18.126210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:18.128265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:18.128562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:18.128954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] PATH_INFO: `/v3/users/96dd7f05388a4c09a04dd4669d2e54d1` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:18.130294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=96dd7f05388a4c09a04dd4669d2e54d1)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:18.132380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:18.200291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:18.331268 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3734d8a1452f4e28a48e747f46d98b29'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.345110 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3734d8a1452f4e28a48e747f46d98b29'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.349665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3734d8a1452f4e28a48e747f46d98b29'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.445557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96dd7f05388a4c09a04dd4669d2e54d1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.446810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96dd7f05388a4c09a04dd4669d2e54d1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.449901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3734d8a1452f4e28a48e747f46d98b29'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.460861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96dd7f05388a4c09a04dd4669d2e54d1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.462337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96dd7f05388a4c09a04dd4669d2e54d1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.472896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3734d8a1452f4e28a48e747f46d98b29'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.480574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 907/1856] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:17 2022] DELETE /identity/v3/users/3734d8a1452f4e28a48e747f46d98b29 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:18.481697 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:18.483944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:18.536750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96dd7f05388a4c09a04dd4669d2e54d1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.567583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 950/1857] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:18 2022] DELETE /identity/v3/users/96dd7f05388a4c09a04dd4669d2e54d1 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:18.568692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:18.573594 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:18.595533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12bf9e-de8d-4641-abe2-e7bbf8605f6c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:18.679433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270934b-816a-43e6-a836-fa14bc2e8d5b admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:18.689503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:18.691578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:18.692109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:18.692990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] PATH_INFO: `/v3/projects/e0ccf73aac884cf29bf66dc92f90768a` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:18.695688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ccf73aac884cf29bf66dc92f90768a)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:18.698936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:18.712601 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de07d91-4296-4a36-8391-497d119cf3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:18.714384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de07d91-4296-4a36-8391-497d119cf3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:18.714840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de07d91-4296-4a36-8391-497d119cf3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:18.715402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de07d91-4296-4a36-8391-497d119cf3e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:18.718752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de07d91-4296-4a36-8391-497d119cf3e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:18.721409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de07d91-4296-4a36-8391-497d119cf3e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:18.725544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de07d91-4296-4a36-8391-497d119cf3e0 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:18.746146 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 908/1858] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:03:18.747662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5b33-b43b-4297-8a74-f64f2223b616 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:18.747944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5b33-b43b-4297-8a74-f64f2223b616 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:18.748248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5b33-b43b-4297-8a74-f64f2223b616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:18.771530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e37f5b33-b43b-4297-8a74-f64f2223b616 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:18.772943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e37f5b33-b43b-4297-8a74-f64f2223b616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:18.782349 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 909/1859] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:18.783268 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f5b33-b43b-4297-8a74-f64f2223b616 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:18.788422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e37f5b33-b43b-4297-8a74-f64f2223b616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:18.792764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:18.813892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f5b33-b43b-4297-8a74-f64f2223b616 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:18.823345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:18.824501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:18.825810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:18.826292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] PATH_INFO: `/v3/projects/1ca16074514c4ce5853357b8c2e4a9d5` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:18.831082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca16074514c4ce5853357b8c2e4a9d5)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:18.834976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:18.861421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ccf73aac884cf29bf66dc92f90768a', 'request_id': 'req-eabce272-0e22-44eb-8c51-fc1a6f7d3508'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.871350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ccf73aac884cf29bf66dc92f90768a', 'request_id': 'req-eabce272-0e22-44eb-8c51-fc1a6f7d3508'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:18.993209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ccf73aac884cf29bf66dc92f90768a', 'request_id': 'req-eabce272-0e22-44eb-8c51-fc1a6f7d3508'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.015841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] The token cache is being invalidate because project e0ccf73aac884cf29bf66dc92f90768a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:19.016451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.018238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.028978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 951/1860] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:18 2022] DELETE /identity/v3/projects/e0ccf73aac884cf29bf66dc92f90768a => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:19.033087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:19.038832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:19.070459 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca16074514c4ce5853357b8c2e4a9d5', 'request_id': 'req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.079041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca16074514c4ce5853357b8c2e4a9d5', 'request_id': 'req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.089440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca16074514c4ce5853357b8c2e4a9d5', 'request_id': 'req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.162893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabce272-0e22-44eb-8c51-fc1a6f7d3508 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:19.242669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b223a9-b687-4b38-9caa-631740bb2f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:19.243838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b223a9-b687-4b38-9caa-631740bb2f0d admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:19.244376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b223a9-b687-4b38-9caa-631740bb2f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:19.244872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b223a9-b687-4b38-9caa-631740bb2f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:19.246059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b223a9-b687-4b38-9caa-631740bb2f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:19.249944 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b223a9-b687-4b38-9caa-631740bb2f0d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:19.257626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] The token cache is being invalidate because project 1ca16074514c4ce5853357b8c2e4a9d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:19.258236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.260969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.260969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 910/1861] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:18 2022] DELETE /identity/v3/projects/1ca16074514c4ce5853357b8c2e4a9d5 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:19.261881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:19.263409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:19.342562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2b7a4-881c-47a8-a283-9d4667ffaf94 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:19.346944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa1463-a9d4-4cae-a27d-8aefb835904e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:19.348121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa1463-a9d4-4cae-a27d-8aefb835904e service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:19.348442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa1463-a9d4-4cae-a27d-8aefb835904e service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:19.348718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa1463-a9d4-4cae-a27d-8aefb835904e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:19.350777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-70fa1463-a9d4-4cae-a27d-8aefb835904e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:19.377018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 952/1862] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:19 2022] POST /identity/v3/projects => generated 383 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:19.377452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcbbb2-3296-4679-92c0-a4c576c479d6 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:19.380826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcbbb2-3296-4679-92c0-a4c576c479d6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:19.380826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcbbb2-3296-4679-92c0-a4c576c479d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:19.381036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa1463-a9d4-4cae-a27d-8aefb835904e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:19.383620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa1463-a9d4-4cae-a27d-8aefb835904e service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:19.405051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 911/1863] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:19.407503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f7dcbbb2-3296-4679-92c0-a4c576c479d6 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:19.408461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04791-654f-406d-b3c9-7920a6929af6 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:19.408990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f7dcbbb2-3296-4679-92c0-a4c576c479d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:19.409517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04791-654f-406d-b3c9-7920a6929af6 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:19.410937 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04791-654f-406d-b3c9-7920a6929af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:19.448728 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-84f04791-654f-406d-b3c9-7920a6929af6 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:19.451210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-84f04791-654f-406d-b3c9-7920a6929af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:19.502698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 953/1864] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:19.508196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcbbb2-3296-4679-92c0-a4c576c479d6 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:19.511222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f7dcbbb2-3296-4679-92c0-a4c576c479d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:19.528091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcbbb2-3296-4679-92c0-a4c576c479d6 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:19.535522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:19.536485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:19.536772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:19.537103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] PATH_INFO: `/v3/projects/fcf3e2e343e04e0888c912a1ef9d9f61` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:19.538728 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcf3e2e343e04e0888c912a1ef9d9f61)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:19.541497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:19.617812 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 912/1865] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:19.618084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04791-654f-406d-b3c9-7920a6929af6 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:19.619775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-84f04791-654f-406d-b3c9-7920a6929af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:19.633056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04791-654f-406d-b3c9-7920a6929af6 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:19.638274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf21d5-3a4e-4e4b-bcf3-05ac5bf8acad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:19.639078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf21d5-3a4e-4e4b-bcf3-05ac5bf8acad admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:19.644904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf21d5-3a4e-4e4b-bcf3-05ac5bf8acad admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:19.644904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf21d5-3a4e-4e4b-bcf3-05ac5bf8acad admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:19.644904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf21d5-3a4e-4e4b-bcf3-05ac5bf8acad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:19.644904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf21d5-3a4e-4e4b-bcf3-05ac5bf8acad admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:19.697087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf3e2e343e04e0888c912a1ef9d9f61', 'request_id': 'req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.705127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf3e2e343e04e0888c912a1ef9d9f61', 'request_id': 'req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.813160 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 913/1866] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:19 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:19.815096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411b4fc-8603-4a1c-a6fa-c47697823f5c None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:19.815501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411b4fc-8603-4a1c-a6fa-c47697823f5c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:19.815914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411b4fc-8603-4a1c-a6fa-c47697823f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:19.840541 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf3e2e343e04e0888c912a1ef9d9f61', 'request_id': 'req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.859857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d411b4fc-8603-4a1c-a6fa-c47697823f5c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:19.863325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d411b4fc-8603-4a1c-a6fa-c47697823f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:19.874604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] The token cache is being invalidate because project fcf3e2e343e04e0888c912a1ef9d9f61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:19.875834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.878056 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d48745f1-37a4-42f7-a2d6-677fb7cc6dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:19.881600 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 954/1867] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:19 2022] DELETE /identity/v3/projects/fcf3e2e343e04e0888c912a1ef9d9f61 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:19.884804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3c77f-80fc-4dea-a70d-f9c73e07dab4 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:19.884804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3c77f-80fc-4dea-a70d-f9c73e07dab4 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:19.885194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3c77f-80fc-4dea-a70d-f9c73e07dab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:19.929845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b2f3c77f-80fc-4dea-a70d-f9c73e07dab4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:19.931363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:19.931902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b2f3c77f-80fc-4dea-a70d-f9c73e07dab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:19.981027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 914/1868] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:19.981027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411b4fc-8603-4a1c-a6fa-c47697823f5c None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:19.983501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d411b4fc-8603-4a1c-a6fa-c47697823f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:20.012118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411b4fc-8603-4a1c-a6fa-c47697823f5c None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:20.021818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1b152-83fc-497c-94d0-3cba3ce1740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:20.022979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1b152-83fc-497c-94d0-3cba3ce1740b admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:20.023548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1b152-83fc-497c-94d0-3cba3ce1740b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:20.024134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1b152-83fc-497c-94d0-3cba3ce1740b admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:20.025396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1b152-83fc-497c-94d0-3cba3ce1740b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:20.028597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1b152-83fc-497c-94d0-3cba3ce1740b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:20.042232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 955/1869] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:20.043593 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3c77f-80fc-4dea-a70d-f9c73e07dab4 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:20.046837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b2f3c77f-80fc-4dea-a70d-f9c73e07dab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:20.056058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 915/1870] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:20.057188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1b152-83fc-497c-94d0-3cba3ce1740b admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:20.071538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3c77f-80fc-4dea-a70d-f9c73e07dab4 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:20.087745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:20.088660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1b152-83fc-497c-94d0-3cba3ce1740b admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:20.088936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:20.089433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:20.089891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] PATH_INFO: `/v3/users/313bfa4722374b62b973f6ea9ff549ac` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:20.091598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] RBAC: Authorizing `identity:delete_user(user_id=313bfa4722374b62b973f6ea9ff549ac)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:20.129916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:20.130079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4ecaa-b334-4564-8bd6-7bf201904d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:20.130079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4ecaa-b334-4564-8bd6-7bf201904d3f admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:20.130079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4ecaa-b334-4564-8bd6-7bf201904d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:20.130079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4ecaa-b334-4564-8bd6-7bf201904d3f admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:20.130658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4ecaa-b334-4564-8bd6-7bf201904d3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:20.130658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4ecaa-b334-4564-8bd6-7bf201904d3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:20.130658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4ecaa-b334-4564-8bd6-7bf201904d3f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:20.155340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 916/1871] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:03:20.155340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4ecaa-b334-4564-8bd6-7bf201904d3f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:20.186339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4ecaa-b334-4564-8bd6-7bf201904d3f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:20.196594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb3e4a-7625-42ee-9595-7f51a6d13d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:20.197653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb3e4a-7625-42ee-9595-7f51a6d13d73 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:20.198328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb3e4a-7625-42ee-9595-7f51a6d13d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:20.198783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb3e4a-7625-42ee-9595-7f51a6d13d73 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:20.199999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb3e4a-7625-42ee-9595-7f51a6d13d73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:20.200709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb3e4a-7625-42ee-9595-7f51a6d13d73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:20.203900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb3e4a-7625-42ee-9595-7f51a6d13d73 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:20.230853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 917/1872] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:03:20.232309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb3e4a-7625-42ee-9595-7f51a6d13d73 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:20.239445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a6cb3e4a-7625-42ee-9595-7f51a6d13d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:20.259192 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb3e4a-7625-42ee-9595-7f51a6d13d73 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:20.266336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:20.266336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:20.266336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:20.266336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] PATH_INFO: `/v3/projects/3fb76f3967594cd1a16c9558af9501fd` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:20.267640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb76f3967594cd1a16c9558af9501fd)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:20.270278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:21.806314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb76f3967594cd1a16c9558af9501fd', 'request_id': 'req-16778203-abe4-4674-9cc6-1cdc047a1a5c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:21.817671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb76f3967594cd1a16c9558af9501fd', 'request_id': 'req-16778203-abe4-4674-9cc6-1cdc047a1a5c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:21.827125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb76f3967594cd1a16c9558af9501fd', 'request_id': 'req-16778203-abe4-4674-9cc6-1cdc047a1a5c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:21.829619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '313bfa4722374b62b973f6ea9ff549ac'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:21.834051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '313bfa4722374b62b973f6ea9ff549ac'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:21.847266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:21.852426 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '313bfa4722374b62b973f6ea9ff549ac'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:21.853840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '313bfa4722374b62b973f6ea9ff549ac'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:21.928059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] The token cache is being invalidate because project 3fb76f3967594cd1a16c9558af9501fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:21.928822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:21.931833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:21.935959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 918/1873] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:20 2022] DELETE /identity/v3/projects/3fb76f3967594cd1a16c9558af9501fd => generated 0 bytes in 1704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:21.941918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:21.944712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:22.012590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '313bfa4722374b62b973f6ea9ff549ac'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:22.061794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 956/1874] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:20 2022] DELETE /identity/v3/users/313bfa4722374b62b973f6ea9ff549ac => generated 0 bytes in 2019 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:22.065546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:22.083001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:22.117852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16778203-abe4-4674-9cc6-1cdc047a1a5c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:22.242784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9bea0-6a5b-4e59-82e5-b88fdaa0835c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:22.254433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0def64-f1a9-47a2-b1f2-50b0fcd4f429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:22.255513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0def64-f1a9-47a2-b1f2-50b0fcd4f429 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:22.256010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0def64-f1a9-47a2-b1f2-50b0fcd4f429 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:22.256526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0def64-f1a9-47a2-b1f2-50b0fcd4f429 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:22.257736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0def64-f1a9-47a2-b1f2-50b0fcd4f429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:22.258241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693063ba-9952-4208-a588-ab5ce8f3e08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:22.259013 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693063ba-9952-4208-a588-ab5ce8f3e08d admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:22.259366 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693063ba-9952-4208-a588-ab5ce8f3e08d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:22.259703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693063ba-9952-4208-a588-ab5ce8f3e08d admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:22.260664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693063ba-9952-4208-a588-ab5ce8f3e08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:22.262276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0def64-f1a9-47a2-b1f2-50b0fcd4f429 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:22.262569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693063ba-9952-4208-a588-ab5ce8f3e08d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:22.277605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 919/1875] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:22.278555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693063ba-9952-4208-a588-ab5ce8f3e08d admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:22.280610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-693063ba-9952-4208-a588-ab5ce8f3e08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:22.299262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693063ba-9952-4208-a588-ab5ce8f3e08d admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:22.306085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebdd0f-be8f-4804-853c-3328fda886a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:22.306821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebdd0f-be8f-4804-853c-3328fda886a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:22.307106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebdd0f-be8f-4804-853c-3328fda886a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:22.307382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebdd0f-be8f-4804-853c-3328fda886a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:22.308083 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebdd0f-be8f-4804-853c-3328fda886a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:22.310104 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebdd0f-be8f-4804-853c-3328fda886a3 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:22.407450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 957/1876] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:22 2022] POST /identity/v3/projects => generated 359 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:22.409318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0def64-f1a9-47a2-b1f2-50b0fcd4f429 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:22.414293 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1b0def64-f1a9-47a2-b1f2-50b0fcd4f429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:22.453674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 920/1877] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:22 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:22.455449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebdd0f-be8f-4804-853c-3328fda886a3 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:22.545031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0def64-f1a9-47a2-b1f2-50b0fcd4f429 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:22.561447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:22.607773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebdd0f-be8f-4804-853c-3328fda886a3 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:22.613672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed903d42-215d-4936-aa7a-a50032366df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:22.614482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed903d42-215d-4936-aa7a-a50032366df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:22.614767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed903d42-215d-4936-aa7a-a50032366df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:22.615038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed903d42-215d-4936-aa7a-a50032366df2 admin admin] PATH_INFO: `/v3/projects/cfbbe24868084fbe85965bafadcbade7/users/f2929afb7adb4f78ad18d7134740466c/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:22.638784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed903d42-215d-4936-aa7a-a50032366df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfbbe24868084fbe85965bafadcbade7, user_id=f2929afb7adb4f78ad18d7134740466c, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:22.641634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed903d42-215d-4936-aa7a-a50032366df2 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:22.680355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:22.681448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:22.682683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:22.683185 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] PATH_INFO: `/v3/users/ba4d5680fbdb4ad9b4ad0e699067b508` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:22.685523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba4d5680fbdb4ad9b4ad0e699067b508)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:22.689397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:22.754177 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 921/1878] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:22 2022] PUT /identity/v3/projects/cfbbe24868084fbe85965bafadcbade7/users/f2929afb7adb4f78ad18d7134740466c/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:22.756817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ada68-2933-495a-9738-d87a2be3a909 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:22.757811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ada68-2933-495a-9738-d87a2be3a909 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:22.757955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ada68-2933-495a-9738-d87a2be3a909 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:22.761718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 922/1879] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:03:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:03:22.763654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ada68-2933-495a-9738-d87a2be3a909 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:22.766882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a05ada68-2933-495a-9738-d87a2be3a909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:22.939933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ada68-2933-495a-9738-d87a2be3a909 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:22.949058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360323a-62a3-4226-94cb-d92284edcc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:22.949950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360323a-62a3-4226-94cb-d92284edcc91 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:22.950390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360323a-62a3-4226-94cb-d92284edcc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:22.951171 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360323a-62a3-4226-94cb-d92284edcc91 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:22.952777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360323a-62a3-4226-94cb-d92284edcc91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:22.955253 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360323a-62a3-4226-94cb-d92284edcc91 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:23.035070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba4d5680fbdb4ad9b4ad0e699067b508'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:23.035936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba4d5680fbdb4ad9b4ad0e699067b508'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:23.045016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba4d5680fbdb4ad9b4ad0e699067b508'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:23.046089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba4d5680fbdb4ad9b4ad0e699067b508'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:23.070701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 923/1880] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:22 2022] POST /identity/v3/users => generated 280 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:23.070701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360323a-62a3-4226-94cb-d92284edcc91 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:23.072608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7360323a-62a3-4226-94cb-d92284edcc91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:23.096676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360323a-62a3-4226-94cb-d92284edcc91 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:23.105010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cf61f-826e-4d3d-986c-afa65cf1a6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:23.106121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf61f-826e-4d3d-986c-afa65cf1a6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:23.106634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf61f-826e-4d3d-986c-afa65cf1a6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:23.107124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf61f-826e-4d3d-986c-afa65cf1a6a5 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:23.108401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cf61f-826e-4d3d-986c-afa65cf1a6a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:23.112529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cf61f-826e-4d3d-986c-afa65cf1a6a5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:23.181205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-432fc946-c5d5-4b0b-b2f5-02f27c23e788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba4d5680fbdb4ad9b4ad0e699067b508'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:23.207758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 958/1881] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:22 2022] DELETE /identity/v3/users/ba4d5680fbdb4ad9b4ad0e699067b508 => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:23.210170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f374-1734-47f0-9f46-c44702672458 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:23.211590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f374-1734-47f0-9f46-c44702672458 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:23.211590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42f374-1734-47f0-9f46-c44702672458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:23.270443 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-6c42f374-1734-47f0-9f46-c44702672458 None None] MFA Rules not processed for user `f2929afb7adb4f78ad18d7134740466c`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:23.277061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6c42f374-1734-47f0-9f46-c44702672458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:23.371720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 924/1882] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:23 2022] POST /identity/v3/users => generated 293 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:23.375100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e1689-fc41-4f3f-8446-c7f6bc745b6d None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:23.376157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e1689-fc41-4f3f-8446-c7f6bc745b6d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:23.376157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e1689-fc41-4f3f-8446-c7f6bc745b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:23.421987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-729e1689-fc41-4f3f-8446-c7f6bc745b6d None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:23.426039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-729e1689-fc41-4f3f-8446-c7f6bc745b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:23.427263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 959/1883] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:23.429484 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f374-1734-47f0-9f46-c44702672458 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:23.433576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6c42f374-1734-47f0-9f46-c44702672458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:23.536784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 925/1884] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:03:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:23.538014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e1689-fc41-4f3f-8446-c7f6bc745b6d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:23.540358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-729e1689-fc41-4f3f-8446-c7f6bc745b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:23.555435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e1689-fc41-4f3f-8446-c7f6bc745b6d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:23.558017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42f374-1734-47f0-9f46-c44702672458 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:23.561013 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd940c-384e-4038-b533-16c5e3bb8d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:23.561640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd940c-384e-4038-b533-16c5e3bb8d18 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:23.561916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd940c-384e-4038-b533-16c5e3bb8d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:23.562355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd940c-384e-4038-b533-16c5e3bb8d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:23.563150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd940c-384e-4038-b533-16c5e3bb8d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:23.563505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:23.564380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:23.564636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:23.564800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd940c-384e-4038-b533-16c5e3bb8d18 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:23.565173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] PATH_INFO: `/v3/projects/17c5623a13f9414e9ece79219ed9eece` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:23.567238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c5623a13f9414e9ece79219ed9eece)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:23.569402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:23.577311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 926/1885] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:23.578196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd940c-384e-4038-b533-16c5e3bb8d18 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:23.590007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd940c-384e-4038-b533-16c5e3bb8d18 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:23.595352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5920dc-a4cf-4945-ac48-1de88a4adb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:23.601302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5920dc-a4cf-4945-ac48-1de88a4adb19 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:23.601302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5920dc-a4cf-4945-ac48-1de88a4adb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:23.601302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5920dc-a4cf-4945-ac48-1de88a4adb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:23.601302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5920dc-a4cf-4945-ac48-1de88a4adb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:23.601302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5920dc-a4cf-4945-ac48-1de88a4adb19 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:23.610520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 927/1886] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:23.611387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5920dc-a4cf-4945-ac48-1de88a4adb19 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:23.613215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3a5920dc-a4cf-4945-ac48-1de88a4adb19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:23.681006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5920dc-a4cf-4945-ac48-1de88a4adb19 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:23.685818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c3029-c09f-494d-accc-f9678808ec96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:23.686103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c3029-c09f-494d-accc-f9678808ec96 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:23.686394 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c3029-c09f-494d-accc-f9678808ec96 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:23.686830 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c3029-c09f-494d-accc-f9678808ec96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:23.703449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c3029-c09f-494d-accc-f9678808ec96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:23.707164 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c3029-c09f-494d-accc-f9678808ec96 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:23.724569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 928/1887] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:23.726032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c3029-c09f-494d-accc-f9678808ec96 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:23.728615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-187c3029-c09f-494d-accc-f9678808ec96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:23.751900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c3029-c09f-494d-accc-f9678808ec96 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:23.760520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8113b-7f47-4f69-ae42-448a16c386a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:23.761559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8113b-7f47-4f69-ae42-448a16c386a0 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:23.762114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8113b-7f47-4f69-ae42-448a16c386a0 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:23.762620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8113b-7f47-4f69-ae42-448a16c386a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:23.775368 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8113b-7f47-4f69-ae42-448a16c386a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:23.777780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8113b-7f47-4f69-ae42-448a16c386a0 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:23.804623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 929/1888] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:23.806884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8113b-7f47-4f69-ae42-448a16c386a0 service nova] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:23.820216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c5623a13f9414e9ece79219ed9eece', 'request_id': 'req-2f8b7077-9711-48d1-b473-d1fee5796ba2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:23.831505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8113b-7f47-4f69-ae42-448a16c386a0 service nova] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:23.833488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c5623a13f9414e9ece79219ed9eece', 'request_id': 'req-2f8b7077-9711-48d1-b473-d1fee5796ba2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:23.964706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5c6e1-78ee-43a1-b663-8ceb295f9759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:23.965615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5c6e1-78ee-43a1-b663-8ceb295f9759 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:23.966050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5c6e1-78ee-43a1-b663-8ceb295f9759 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:23.966429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5c6e1-78ee-43a1-b663-8ceb295f9759 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:23.967683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5c6e1-78ee-43a1-b663-8ceb295f9759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:23.970270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5c6e1-78ee-43a1-b663-8ceb295f9759 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:23.979548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:24.003530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 930/1889] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:24.005784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5c6e1-78ee-43a1-b663-8ceb295f9759 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:24.009998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c5623a13f9414e9ece79219ed9eece', 'request_id': 'req-2f8b7077-9711-48d1-b473-d1fee5796ba2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:24.021102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5c6e1-78ee-43a1-b663-8ceb295f9759 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:24.028868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095587a-71c0-4eed-b5a5-31a24b280517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:24.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095587a-71c0-4eed-b5a5-31a24b280517 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:24.030546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095587a-71c0-4eed-b5a5-31a24b280517 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:24.031055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095587a-71c0-4eed-b5a5-31a24b280517 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:24.032293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095587a-71c0-4eed-b5a5-31a24b280517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:24.035373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095587a-71c0-4eed-b5a5-31a24b280517 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:24.039885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] The token cache is being invalidate because project 17c5623a13f9414e9ece79219ed9eece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:24.041270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:24.042422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:24.044670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 960/1890] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:23 2022] DELETE /identity/v3/projects/17c5623a13f9414e9ece79219ed9eece => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:24.045861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:24.048333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:24.058836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 931/1891] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:24.061210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60761f18-2fef-4984-8252-f01243ddfa28 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:24.061670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60761f18-2fef-4984-8252-f01243ddfa28 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:24.062119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60761f18-2fef-4984-8252-f01243ddfa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:24.071244 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b7077-9711-48d1-b473-d1fee5796ba2 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:24.079289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f6ec2-3355-4724-aa71-a3dbf0f55726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:24.080396 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f6ec2-3355-4724-aa71-a3dbf0f55726 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:24.081016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f6ec2-3355-4724-aa71-a3dbf0f55726 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:24.085684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f6ec2-3355-4724-aa71-a3dbf0f55726 admin admin] PATH_INFO: `/v3/projects/bd55d912da7c4280879a3bf5be662439/users/14d8f33ad6db4b0abfa8d242cbf06b7b/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:24.100278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-60761f18-2fef-4984-8252-f01243ddfa28 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:24.102726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-60761f18-2fef-4984-8252-f01243ddfa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:24.108147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f6ec2-3355-4724-aa71-a3dbf0f55726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd55d912da7c4280879a3bf5be662439, user_id=14d8f33ad6db4b0abfa8d242cbf06b7b, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:24.112360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f6ec2-3355-4724-aa71-a3dbf0f55726 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:24.132214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 932/1892] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:24.133604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60761f18-2fef-4984-8252-f01243ddfa28 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:24.135948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-60761f18-2fef-4984-8252-f01243ddfa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:24.155944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60761f18-2fef-4984-8252-f01243ddfa28 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:24.168691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5d3f5-cf24-49b5-925b-06b23e0f3e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:24.170337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5d3f5-cf24-49b5-925b-06b23e0f3e3c admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:24.170833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5d3f5-cf24-49b5-925b-06b23e0f3e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:24.171391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5d3f5-cf24-49b5-925b-06b23e0f3e3c admin admin] PATH_INFO: `/v3/projects/9ede7129ace74745b56f13bf58a590ed/users/359219acf33d4801ba241c6c222fb902/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:24.221763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5d3f5-cf24-49b5-925b-06b23e0f3e3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ede7129ace74745b56f13bf58a590ed, user_id=359219acf33d4801ba241c6c222fb902, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:24.226622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5d3f5-cf24-49b5-925b-06b23e0f3e3c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:24.272602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 961/1893] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:24 2022] PUT /identity/v3/projects/bd55d912da7c4280879a3bf5be662439/users/14d8f33ad6db4b0abfa8d242cbf06b7b/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:24.276691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf2ca1-d548-43c7-9520-4c03ffd578a3 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:24.277183 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf2ca1-d548-43c7-9520-4c03ffd578a3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:24.277667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf2ca1-d548-43c7-9520-4c03ffd578a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:24.319009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-3ccf2ca1-d548-43c7-9520-4c03ffd578a3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:24.322780 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3ccf2ca1-d548-43c7-9520-4c03ffd578a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:24.381205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 933/1894] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:24 2022] PUT /identity/v3/projects/9ede7129ace74745b56f13bf58a590ed/users/359219acf33d4801ba241c6c222fb902/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:24.383093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5d3f5-cf24-49b5-925b-06b23e0f3e3c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:24.387190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-87e5d3f5-cf24-49b5-925b-06b23e0f3e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:24.425986 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 962/1895] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:24.428458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc770f-6be3-4a3f-856b-f3b1c1df6ab8 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:24.428751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc770f-6be3-4a3f-856b-f3b1c1df6ab8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:24.434123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc770f-6be3-4a3f-856b-f3b1c1df6ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:24.470246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e0bc770f-6be3-4a3f-856b-f3b1c1df6ab8 None None] MFA Rules not processed for user `14d8f33ad6db4b0abfa8d242cbf06b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:24.471812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e0bc770f-6be3-4a3f-856b-f3b1c1df6ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:24.486525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5d3f5-cf24-49b5-925b-06b23e0f3e3c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:24.495201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:24.495918 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:24.496234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:24.496522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] PATH_INFO: `/v3/projects/8cd42695272b495ca48a7f9608ef1c9f` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:24.498296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd42695272b495ca48a7f9608ef1c9f)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:24.501402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:24.621216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 963/1896] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:24.624653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008e4a-d0c8-48ae-95b9-2858dc1e8e58 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:24.626006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008e4a-d0c8-48ae-95b9-2858dc1e8e58 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:24.626746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008e4a-d0c8-48ae-95b9-2858dc1e8e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:24.682318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd42695272b495ca48a7f9608ef1c9f', 'request_id': 'req-26e69656-6553-41de-a7fd-6484d01826d3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:24.684710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1e008e4a-d0c8-48ae-95b9-2858dc1e8e58 None None] MFA Rules not processed for user `359219acf33d4801ba241c6c222fb902`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:24.686116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1e008e4a-d0c8-48ae-95b9-2858dc1e8e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:24.700723 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd42695272b495ca48a7f9608ef1c9f', 'request_id': 'req-26e69656-6553-41de-a7fd-6484d01826d3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:24.714927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd42695272b495ca48a7f9608ef1c9f', 'request_id': 'req-26e69656-6553-41de-a7fd-6484d01826d3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:24.781787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 964/1897] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:24.787591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e008e4a-d0c8-48ae-95b9-2858dc1e8e58 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:24.861039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] The token cache is being invalidate because project 8cd42695272b495ca48a7f9608ef1c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:24.861498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:24.862516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:24.865810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 934/1898] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:24 2022] DELETE /identity/v3/projects/8cd42695272b495ca48a7f9608ef1c9f => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:24.867712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:24.870899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:24.958523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e008e4a-d0c8-48ae-95b9-2858dc1e8e58 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:24.966598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b554e-0e8c-4f3f-a01f-cb50d3d5f7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:24.967451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b554e-0e8c-4f3f-a01f-cb50d3d5f7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:24.968011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b554e-0e8c-4f3f-a01f-cb50d3d5f7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:24.968483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b554e-0e8c-4f3f-a01f-cb50d3d5f7d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:24.969200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b554e-0e8c-4f3f-a01f-cb50d3d5f7d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:24.969588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b554e-0e8c-4f3f-a01f-cb50d3d5f7d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:24.971747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b554e-0e8c-4f3f-a01f-cb50d3d5f7d7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:25.004472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 965/1899] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:03:25.014685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b554e-0e8c-4f3f-a01f-cb50d3d5f7d7 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:25.016356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e69656-6553-41de-a7fd-6484d01826d3 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:25.024560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-885b554e-0e8c-4f3f-a01f-cb50d3d5f7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:25.026401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ca2a4-59b2-42a8-b311-24e610f4559c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:25.027721 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ca2a4-59b2-42a8-b311-24e610f4559c service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.028641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ca2a4-59b2-42a8-b311-24e610f4559c service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.029388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ca2a4-59b2-42a8-b311-24e610f4559c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:25.033274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-724ca2a4-59b2-42a8-b311-24e610f4559c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:25.039409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b554e-0e8c-4f3f-a01f-cb50d3d5f7d7 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:25.046730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028a78b-0347-4d84-815b-d1db7a5d0704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:25.047525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028a78b-0347-4d84-815b-d1db7a5d0704 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.047886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028a78b-0347-4d84-815b-d1db7a5d0704 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.048344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028a78b-0347-4d84-815b-d1db7a5d0704 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:25.050749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028a78b-0347-4d84-815b-d1db7a5d0704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:25.051817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028a78b-0347-4d84-815b-d1db7a5d0704 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:25.162228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ca2a4-59b2-42a8-b311-24e610f4559c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:25.163754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ca2a4-59b2-42a8-b311-24e610f4559c service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:25.177718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 935/1900] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:24 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:25.179398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820a30d-b379-45e8-89a2-c4c1c7e65a4c None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.179813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820a30d-b379-45e8-89a2-c4c1c7e65a4c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.180114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820a30d-b379-45e8-89a2-c4c1c7e65a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:25.192737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 966/1901] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:25 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:25.194558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028a78b-0347-4d84-815b-d1db7a5d0704 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:25.203823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-8820a30d-b379-45e8-89a2-c4c1c7e65a4c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:25.204667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-0028a78b-0347-4d84-815b-d1db7a5d0704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:25.205105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8820a30d-b379-45e8-89a2-c4c1c7e65a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:25.227252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:25.290900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 936/1902] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:25.291852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820a30d-b379-45e8-89a2-c4c1c7e65a4c None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:25.310315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820a30d-b379-45e8-89a2-c4c1c7e65a4c None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:25.315396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca32a3-7e56-4117-ba39-8d2f628e39a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:25.316305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca32a3-7e56-4117-ba39-8d2f628e39a9 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.316589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca32a3-7e56-4117-ba39-8d2f628e39a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.316883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca32a3-7e56-4117-ba39-8d2f628e39a9 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:25.317547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca32a3-7e56-4117-ba39-8d2f628e39a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:25.319548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca32a3-7e56-4117-ba39-8d2f628e39a9 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:25.336810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028a78b-0347-4d84-815b-d1db7a5d0704 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:25.343794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48340e35-adf5-4731-9864-b56d442734d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:25.346383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48340e35-adf5-4731-9864-b56d442734d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.346383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48340e35-adf5-4731-9864-b56d442734d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.346383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48340e35-adf5-4731-9864-b56d442734d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:25.346958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48340e35-adf5-4731-9864-b56d442734d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:25.351188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48340e35-adf5-4731-9864-b56d442734d0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:25.450430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 937/1903] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:25 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:25.450430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca32a3-7e56-4117-ba39-8d2f628e39a9 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:25.465020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-07ca32a3-7e56-4117-ba39-8d2f628e39a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:25.489233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 967/1904] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:25 2022] POST /identity/v3/projects => generated 383 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:25.491795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd55b3-d290-4d67-88e6-65cded423693 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.494033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd55b3-d290-4d67-88e6-65cded423693 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.494598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd55b3-d290-4d67-88e6-65cded423693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:25.496048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca32a3-7e56-4117-ba39-8d2f628e39a9 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:25.549068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-fbbd55b3-d290-4d67-88e6-65cded423693 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:25.552873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fbbd55b3-d290-4d67-88e6-65cded423693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:25.648883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:25.649677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.649972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.650242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] PATH_INFO: `/v3/users/14d8f33ad6db4b0abfa8d242cbf06b7b` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:25.651772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d8f33ad6db4b0abfa8d242cbf06b7b)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:25.654725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:25.671022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 968/1905] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:25.671658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd55b3-d290-4d67-88e6-65cded423693 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:25.686927 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd55b3-d290-4d67-88e6-65cded423693 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:25.693282 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a8138-7192-494a-85ae-ea80919a3ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:25.693880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a8138-7192-494a-85ae-ea80919a3ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.694236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a8138-7192-494a-85ae-ea80919a3ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.694576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a8138-7192-494a-85ae-ea80919a3ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:25.695414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a8138-7192-494a-85ae-ea80919a3ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:25.697258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a8138-7192-494a-85ae-ea80919a3ff3 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:25.712034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 969/1906] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:25.714921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a8138-7192-494a-85ae-ea80919a3ff3 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:25.729536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a8138-7192-494a-85ae-ea80919a3ff3 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:25.735485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce3353-5b0a-4be0-a144-a59b1f433059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:25.736122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce3353-5b0a-4be0-a144-a59b1f433059 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.736467 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce3353-5b0a-4be0-a144-a59b1f433059 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.736788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce3353-5b0a-4be0-a144-a59b1f433059 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:25.737444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce3353-5b0a-4be0-a144-a59b1f433059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:25.740216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce3353-5b0a-4be0-a144-a59b1f433059 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:25.896566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 970/1907] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:25 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:25.897439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce3353-5b0a-4be0-a144-a59b1f433059 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:25.918785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce3353-5b0a-4be0-a144-a59b1f433059 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:25.928267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df7adf-712b-4c23-9bda-1a28c54c0e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:25.929329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7adf-712b-4c23-9bda-1a28c54c0e1a admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.929940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7adf-712b-4c23-9bda-1a28c54c0e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.930601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7adf-712b-4c23-9bda-1a28c54c0e1a admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:25.932068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df7adf-712b-4c23-9bda-1a28c54c0e1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:25.932749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df7adf-712b-4c23-9bda-1a28c54c0e1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:25.936485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df7adf-712b-4c23-9bda-1a28c54c0e1a admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:25.958312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d8f33ad6db4b0abfa8d242cbf06b7b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:25.968729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 971/1908] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:03:25.968729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df7adf-712b-4c23-9bda-1a28c54c0e1a admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:25.973025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d8f33ad6db4b0abfa8d242cbf06b7b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:25.973887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d8f33ad6db4b0abfa8d242cbf06b7b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:25.990770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df7adf-712b-4c23-9bda-1a28c54c0e1a admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:25.998953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53230c38-b955-4ace-bdc6-77bf51416884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:25.999178 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53230c38-b955-4ace-bdc6-77bf51416884 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:25.999599 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53230c38-b955-4ace-bdc6-77bf51416884 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:25.999871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53230c38-b955-4ace-bdc6-77bf51416884 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:26.000723 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53230c38-b955-4ace-bdc6-77bf51416884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:26.002493 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53230c38-b955-4ace-bdc6-77bf51416884 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:26.016823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 972/1909] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:26.017848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53230c38-b955-4ace-bdc6-77bf51416884 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:26.024492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-53230c38-b955-4ace-bdc6-77bf51416884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:26.041914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53230c38-b955-4ace-bdc6-77bf51416884 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:26.050693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9d628-5742-4589-945b-fce5a5646311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:26.051707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9d628-5742-4589-945b-fce5a5646311 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:26.052436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9d628-5742-4589-945b-fce5a5646311 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:26.053031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9d628-5742-4589-945b-fce5a5646311 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:26.053952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9d628-5742-4589-945b-fce5a5646311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:26.057730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9d628-5742-4589-945b-fce5a5646311 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:26.072888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d8f33ad6db4b0abfa8d242cbf06b7b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:26.099210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 973/1910] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:26.099210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9d628-5742-4589-945b-fce5a5646311 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:26.100145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-71c9d628-5742-4589-945b-fce5a5646311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:26.117839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d8f33ad6db4b0abfa8d242cbf06b7b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:26.118907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9d628-5742-4589-945b-fce5a5646311 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:26.126626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5f4b0-cacf-4000-b42a-c237a5a6bcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:26.127482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5f4b0-cacf-4000-b42a-c237a5a6bcf1 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:26.129084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5f4b0-cacf-4000-b42a-c237a5a6bcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:26.129569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5f4b0-cacf-4000-b42a-c237a5a6bcf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:26.130407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5f4b0-cacf-4000-b42a-c237a5a6bcf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:26.133868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 938/1911] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:25 2022] DELETE /identity/v3/users/14d8f33ad6db4b0abfa8d242cbf06b7b => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:26.147583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:26.147583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:26.149335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5f4b0-cacf-4000-b42a-c237a5a6bcf1 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:26.293313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abada4-7b2a-4950-a783-620f39e6e64b admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:26.299508 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681c2f5-60c6-4e87-ad29-0f6e294a05ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:26.301198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681c2f5-60c6-4e87-ad29-0f6e294a05ed admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:26.301198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681c2f5-60c6-4e87-ad29-0f6e294a05ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:26.301529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681c2f5-60c6-4e87-ad29-0f6e294a05ed admin admin] PATH_INFO: `/v3/projects/1d28adfb08a741638ea9702916b24194/users/a615dae8e5d946c88e5f13238a3c192e/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:26.320263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681c2f5-60c6-4e87-ad29-0f6e294a05ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d28adfb08a741638ea9702916b24194, user_id=a615dae8e5d946c88e5f13238a3c192e, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:26.323269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681c2f5-60c6-4e87-ad29-0f6e294a05ed admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:26.347266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 974/1912] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:26 2022] POST /identity/v3/projects => generated 381 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:26.348568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5f4b0-cacf-4000-b42a-c237a5a6bcf1 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:26.446484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 939/1913] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:26 2022] PUT /identity/v3/projects/1d28adfb08a741638ea9702916b24194/users/a615dae8e5d946c88e5f13238a3c192e/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:26.446484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3277a-6b3e-4baa-9b84-ac1da5729eba None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:26.446484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3277a-6b3e-4baa-9b84-ac1da5729eba None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:26.446484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3277a-6b3e-4baa-9b84-ac1da5729eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:26.447069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5f4b0-cacf-4000-b42a-c237a5a6bcf1 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:26.486496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-45b3277a-6b3e-4baa-9b84-ac1da5729eba None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:26.489574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-45b3277a-6b3e-4baa-9b84-ac1da5729eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:26.531640 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60a3bb-e283-4ad3-b1b4-34d118b9fd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:26.532323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60a3bb-e283-4ad3-b1b4-34d118b9fd0b admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:26.532609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60a3bb-e283-4ad3-b1b4-34d118b9fd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:26.532890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60a3bb-e283-4ad3-b1b4-34d118b9fd0b admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:26.533549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60a3bb-e283-4ad3-b1b4-34d118b9fd0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:26.535759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60a3bb-e283-4ad3-b1b4-34d118b9fd0b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:26.554623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 975/1914] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:26.554623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60a3bb-e283-4ad3-b1b4-34d118b9fd0b admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:26.573191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60a3bb-e283-4ad3-b1b4-34d118b9fd0b admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:26.574263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 940/1915] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:26.575604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d1879-39ce-42da-a5de-8eea38ced2c1 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:26.575861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d1879-39ce-42da-a5de-8eea38ced2c1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:26.578961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d1879-39ce-42da-a5de-8eea38ced2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:26.586202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba066c-6f2a-4446-8d17-1caddd8399fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:26.587278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba066c-6f2a-4446-8d17-1caddd8399fd admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:26.587770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba066c-6f2a-4446-8d17-1caddd8399fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:26.588322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba066c-6f2a-4446-8d17-1caddd8399fd admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:26.589457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba066c-6f2a-4446-8d17-1caddd8399fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:26.592412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba066c-6f2a-4446-8d17-1caddd8399fd admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:26.598920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:26.638835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d58d1879-39ce-42da-a5de-8eea38ced2c1 None None] MFA Rules not processed for user `a615dae8e5d946c88e5f13238a3c192e`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:26.640786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d58d1879-39ce-42da-a5de-8eea38ced2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:26.760907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 941/1916] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:26.761782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d1879-39ce-42da-a5de-8eea38ced2c1 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:26.769782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 976/1917] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:26 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:26.769782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba066c-6f2a-4446-8d17-1caddd8399fd admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:26.777978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-15ba066c-6f2a-4446-8d17-1caddd8399fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:26.780412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d1879-39ce-42da-a5de-8eea38ced2c1 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:26.802349 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71a68c-ec1c-4f07-8426-00308ffa7ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:26.803047 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71a68c-ec1c-4f07-8426-00308ffa7ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:26.803346 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71a68c-ec1c-4f07-8426-00308ffa7ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:26.803645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71a68c-ec1c-4f07-8426-00308ffa7ea2 admin admin] PATH_INFO: `/v3/projects/24332f4006014757a7eda8bfb5d7ad8d/users/5ff987ea8e0a485b9f88fcd365269d33/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:26.827455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71a68c-ec1c-4f07-8426-00308ffa7ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24332f4006014757a7eda8bfb5d7ad8d, user_id=5ff987ea8e0a485b9f88fcd365269d33, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:26.830245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71a68c-ec1c-4f07-8426-00308ffa7ea2 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:26.902844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba066c-6f2a-4446-8d17-1caddd8399fd admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:26.907966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff56604-0eb3-4160-9544-f8bdb43b9c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:26.908742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff56604-0eb3-4160-9544-f8bdb43b9c3e service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:26.909019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff56604-0eb3-4160-9544-f8bdb43b9c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:26.909364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff56604-0eb3-4160-9544-f8bdb43b9c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:26.926491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff56604-0eb3-4160-9544-f8bdb43b9c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:26.928077 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff56604-0eb3-4160-9544-f8bdb43b9c3e service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:26.952077 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 977/1918] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:26.953681 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff56604-0eb3-4160-9544-f8bdb43b9c3e service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:26.975346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff56604-0eb3-4160-9544-f8bdb43b9c3e service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:26.986449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 942/1919] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:26 2022] PUT /identity/v3/projects/24332f4006014757a7eda8bfb5d7ad8d/users/5ff987ea8e0a485b9f88fcd365269d33/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:26.989989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71a68c-ec1c-4f07-8426-00308ffa7ea2 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:27.130565 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71a68c-ec1c-4f07-8426-00308ffa7ea2 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:27.138235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941a970-6095-40de-a91f-96478639b59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:27.139089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941a970-6095-40de-a91f-96478639b59a admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.140006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941a970-6095-40de-a91f-96478639b59a admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.140319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941a970-6095-40de-a91f-96478639b59a admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.141045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941a970-6095-40de-a91f-96478639b59a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:27.143104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941a970-6095-40de-a91f-96478639b59a admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:27.148025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af0f3a-024d-4854-9455-5ace27a09168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:27.149145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0f3a-024d-4854-9455-5ace27a09168 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.149610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0f3a-024d-4854-9455-5ace27a09168 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.150084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0f3a-024d-4854-9455-5ace27a09168 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.151276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af0f3a-024d-4854-9455-5ace27a09168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:27.154472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af0f3a-024d-4854-9455-5ace27a09168 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:27.175750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 943/1920] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:27.177807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c80d-4800-4ca1-afc1-c9a12964870a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.178310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c80d-4800-4ca1-afc1-c9a12964870a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.179132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c80d-4800-4ca1-afc1-c9a12964870a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.229593 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-0fe3c80d-4800-4ca1-afc1-c9a12964870a None None] MFA Rules not processed for user `5ff987ea8e0a485b9f88fcd365269d33`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:27.231744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0fe3c80d-4800-4ca1-afc1-c9a12964870a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:27.325814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 944/1921] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:27.326714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3c80d-4800-4ca1-afc1-c9a12964870a None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:27.341635 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3c80d-4800-4ca1-afc1-c9a12964870a None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:27.346619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408ea0-48b7-4f40-a96e-a03918f35bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:27.347795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408ea0-48b7-4f40-a96e-a03918f35bfb admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.348073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408ea0-48b7-4f40-a96e-a03918f35bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.348386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408ea0-48b7-4f40-a96e-a03918f35bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.349238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1408ea0-48b7-4f40-a96e-a03918f35bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:27.350751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1408ea0-48b7-4f40-a96e-a03918f35bfb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:27.366042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 945/1922] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:27.367519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408ea0-48b7-4f40-a96e-a03918f35bfb admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:27.387613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408ea0-48b7-4f40-a96e-a03918f35bfb admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:27.395847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ae5a9-e978-4575-91d0-94c78bdef24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:27.397074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ae5a9-e978-4575-91d0-94c78bdef24c admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.397569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ae5a9-e978-4575-91d0-94c78bdef24c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.398059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ae5a9-e978-4575-91d0-94c78bdef24c admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.399221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ae5a9-e978-4575-91d0-94c78bdef24c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:27.405619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ae5a9-e978-4575-91d0-94c78bdef24c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:27.419793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 978/1923] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:26 2022] POST /identity/v3/projects => generated 387 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:27.421370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941a970-6095-40de-a91f-96478639b59a admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:27.557500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 946/1924] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:27 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:27.559365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ae5a9-e978-4575-91d0-94c78bdef24c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:27.562964 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941a970-6095-40de-a91f-96478639b59a admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:27.641139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03207-5e83-40f1-9d80-d6aed9507a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:27.642131 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03207-5e83-40f1-9d80-d6aed9507a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.642581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03207-5e83-40f1-9d80-d6aed9507a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.643640 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03207-5e83-40f1-9d80-d6aed9507a7d admin admin] PATH_INFO: `/v3/projects/658f63649667484cbd7a11de765e5bd3/users/49a32a1453f34563934bec7ffe9f4346/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.648831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ae5a9-e978-4575-91d0-94c78bdef24c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:27.655252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aabc8-f58a-4322-bc0f-c95319dea95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:27.655865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aabc8-f58a-4322-bc0f-c95319dea95f admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.656143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aabc8-f58a-4322-bc0f-c95319dea95f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.656448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aabc8-f58a-4322-bc0f-c95319dea95f admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.657092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aabc8-f58a-4322-bc0f-c95319dea95f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:27.659124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aabc8-f58a-4322-bc0f-c95319dea95f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:27.664525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df03207-5e83-40f1-9d80-d6aed9507a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=658f63649667484cbd7a11de765e5bd3, user_id=49a32a1453f34563934bec7ffe9f4346, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:27.670215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df03207-5e83-40f1-9d80-d6aed9507a7d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:27.764770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 947/1925] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:27 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:27.765999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aabc8-f58a-4322-bc0f-c95319dea95f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:27.777313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aabc8-f58a-4322-bc0f-c95319dea95f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:27.782860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14816bfa-cd8e-458b-97da-3243cce24295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:27.783614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14816bfa-cd8e-458b-97da-3243cce24295 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.784003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14816bfa-cd8e-458b-97da-3243cce24295 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.784404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14816bfa-cd8e-458b-97da-3243cce24295 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.785815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14816bfa-cd8e-458b-97da-3243cce24295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:27.787930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14816bfa-cd8e-458b-97da-3243cce24295 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:27.805416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 979/1926] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:27 2022] PUT /identity/v3/projects/658f63649667484cbd7a11de765e5bd3/users/49a32a1453f34563934bec7ffe9f4346/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:27.806266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03207-5e83-40f1-9d80-d6aed9507a7d admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:27.897775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03207-5e83-40f1-9d80-d6aed9507a7d admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:27.905999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838e5cd-aeee-452a-86b6-e77edac25232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:27.907055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838e5cd-aeee-452a-86b6-e77edac25232 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.907565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838e5cd-aeee-452a-86b6-e77edac25232 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.908078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838e5cd-aeee-452a-86b6-e77edac25232 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.909235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838e5cd-aeee-452a-86b6-e77edac25232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:27.913153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838e5cd-aeee-452a-86b6-e77edac25232 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:27.934017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 980/1927] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:27.935991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838e5cd-aeee-452a-86b6-e77edac25232 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:27.939239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-0838e5cd-aeee-452a-86b6-e77edac25232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:27.955613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 948/1928] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:27 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:27.958178 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c2dfe-0128-4e99-ba59-868772fc8d64 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.958558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c2dfe-0128-4e99-ba59-868772fc8d64 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.958744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c2dfe-0128-4e99-ba59-868772fc8d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.969747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838e5cd-aeee-452a-86b6-e77edac25232 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:27.979469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:27.987285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:27.987285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:27.987285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] PATH_INFO: `/v3/projects/bd55d912da7c4280879a3bf5be662439` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:27.987285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd55d912da7c4280879a3bf5be662439)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:27.987285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:27.998638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-708c2dfe-0128-4e99-ba59-868772fc8d64 None None] MFA Rules not processed for user `49a32a1453f34563934bec7ffe9f4346`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:28.000136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-708c2dfe-0128-4e99-ba59-868772fc8d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:28.102750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 949/1929] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:28.107938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c2dfe-0128-4e99-ba59-868772fc8d64 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:28.121699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c2dfe-0128-4e99-ba59-868772fc8d64 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:28.127205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852624-8feb-44a6-a7ca-9839a3b3c733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:28.128986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852624-8feb-44a6-a7ca-9839a3b3c733 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:28.129141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852624-8feb-44a6-a7ca-9839a3b3c733 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:28.129342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852624-8feb-44a6-a7ca-9839a3b3c733 admin admin] PATH_INFO: `/v3/projects/b2425c01d8014202b624ce72c24b8baa/users/6c5c0983061d423690d951643b92a5b5/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:28.157947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0852624-8feb-44a6-a7ca-9839a3b3c733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2425c01d8014202b624ce72c24b8baa, user_id=6c5c0983061d423690d951643b92a5b5, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:28.167381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0852624-8feb-44a6-a7ca-9839a3b3c733 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:28.258346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:28.266162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 950/1930] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:28 2022] PUT /identity/v3/projects/b2425c01d8014202b624ce72c24b8baa/users/6c5c0983061d423690d951643b92a5b5/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:28.269747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852624-8feb-44a6-a7ca-9839a3b3c733 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:28.280111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd55d912da7c4280879a3bf5be662439', 'request_id': 'req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:28.288888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd55d912da7c4280879a3bf5be662439', 'request_id': 'req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:28.347707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852624-8feb-44a6-a7ca-9839a3b3c733 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:28.368394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd55d912da7c4280879a3bf5be662439', 'request_id': 'req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:28.385603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] The token cache is being invalidate because project bd55d912da7c4280879a3bf5be662439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:28.389002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:28.389002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:28.389002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 981/1931] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:27 2022] DELETE /identity/v3/projects/bd55d912da7c4280879a3bf5be662439 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:28.393221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:28.396462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:28.468619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61025152-d23b-4b63-9a8c-da1331ae7e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:28.473295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61025152-d23b-4b63-9a8c-da1331ae7e17 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:28.473295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61025152-d23b-4b63-9a8c-da1331ae7e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:28.473295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61025152-d23b-4b63-9a8c-da1331ae7e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:28.473295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61025152-d23b-4b63-9a8c-da1331ae7e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:28.473295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61025152-d23b-4b63-9a8c-da1331ae7e17 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:28.487336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 951/1932] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:28.489082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61025152-d23b-4b63-9a8c-da1331ae7e17 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:28.491402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-61025152-d23b-4b63-9a8c-da1331ae7e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:28.498141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:28.511958 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61025152-d23b-4b63-9a8c-da1331ae7e17 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:28.517271 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36270-3c63-4ff5-8766-0c5157c29696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:28.518308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36270-3c63-4ff5-8766-0c5157c29696 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:28.518725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36270-3c63-4ff5-8766-0c5157c29696 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:28.519447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36270-3c63-4ff5-8766-0c5157c29696 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:28.520373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a36270-3c63-4ff5-8766-0c5157c29696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:28.522548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a36270-3c63-4ff5-8766-0c5157c29696 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:28.538622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 952/1933] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:28.538889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130c3d5-90f4-4102-b7e7-2ea03c27a2f6 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:28.539018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fcb03-d583-402f-8c41-45cfe15d10cc None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:28.539018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fcb03-d583-402f-8c41-45cfe15d10cc None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:28.539018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fcb03-d583-402f-8c41-45cfe15d10cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:28.549128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22001a65-0727-4df7-9f75-f55a5a30a851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:28.550790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22001a65-0727-4df7-9f75-f55a5a30a851 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:28.551106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22001a65-0727-4df7-9f75-f55a5a30a851 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:28.551416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22001a65-0727-4df7-9f75-f55a5a30a851 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:28.552039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22001a65-0727-4df7-9f75-f55a5a30a851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:28.554012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22001a65-0727-4df7-9f75-f55a5a30a851 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:28.576910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-916fcb03-d583-402f-8c41-45cfe15d10cc None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:28.576910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-916fcb03-d583-402f-8c41-45cfe15d10cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:28.588229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 953/1934] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:28.588229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fcb03-d583-402f-8c41-45cfe15d10cc None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:28.589783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-916fcb03-d583-402f-8c41-45cfe15d10cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:28.603348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fcb03-d583-402f-8c41-45cfe15d10cc None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:28.608446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d16ca-27b3-4d48-a67e-ca4abf026aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:28.609125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d16ca-27b3-4d48-a67e-ca4abf026aad admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:28.609409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d16ca-27b3-4d48-a67e-ca4abf026aad admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:28.609712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d16ca-27b3-4d48-a67e-ca4abf026aad admin admin] PATH_INFO: `/v3/projects/dd2e1cfdfc2f4efba65aea6be02e456b/users/e30069ef241647699f41cbd8588c279d/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:28.626905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d16ca-27b3-4d48-a67e-ca4abf026aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd2e1cfdfc2f4efba65aea6be02e456b, user_id=e30069ef241647699f41cbd8588c279d, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:28.630236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d16ca-27b3-4d48-a67e-ca4abf026aad admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:28.714404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 954/1935] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:28 2022] PUT /identity/v3/projects/dd2e1cfdfc2f4efba65aea6be02e456b/users/e30069ef241647699f41cbd8588c279d/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:28.715949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d16ca-27b3-4d48-a67e-ca4abf026aad admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:28.720329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 982/1936] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:28 2022] POST /identity/v3/projects => generated 381 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:28.720329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6c52-e067-44ed-b199-998c4332bbfb None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:28.720696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6c52-e067-44ed-b199-998c4332bbfb None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:28.721770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6c52-e067-44ed-b199-998c4332bbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:28.746036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-763a6c52-e067-44ed-b199-998c4332bbfb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:28.747281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-763a6c52-e067-44ed-b199-998c4332bbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:28.817333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d16ca-27b3-4d48-a67e-ca4abf026aad admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:28.841613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 983/1937] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:28.843161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a6c52-e067-44ed-b199-998c4332bbfb None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:28.855690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a6c52-e067-44ed-b199-998c4332bbfb None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:28.860790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef2814-0688-43ef-b434-dd19b0b88f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:28.861417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef2814-0688-43ef-b434-dd19b0b88f66 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:28.861696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef2814-0688-43ef-b434-dd19b0b88f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:28.861965 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef2814-0688-43ef-b434-dd19b0b88f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:28.862676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef2814-0688-43ef-b434-dd19b0b88f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:28.864360 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef2814-0688-43ef-b434-dd19b0b88f66 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:28.877385 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 984/1938] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:28.879540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef2814-0688-43ef-b434-dd19b0b88f66 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:28.890935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef2814-0688-43ef-b434-dd19b0b88f66 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:28.901967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1f1f1-78cc-4955-8877-6889acfa82a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:28.903044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1f1f1-78cc-4955-8877-6889acfa82a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:28.903557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1f1f1-78cc-4955-8877-6889acfa82a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:28.904072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1f1f1-78cc-4955-8877-6889acfa82a7 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:28.905410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1f1f1-78cc-4955-8877-6889acfa82a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:28.909385 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1f1f1-78cc-4955-8877-6889acfa82a7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:28.910297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29915a5e-9751-419a-bac7-1c259346bfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:28.911016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29915a5e-9751-419a-bac7-1c259346bfcb admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:28.911293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29915a5e-9751-419a-bac7-1c259346bfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:28.916786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29915a5e-9751-419a-bac7-1c259346bfcb admin admin] PATH_INFO: `/v3/projects/b2425c01d8014202b624ce72c24b8baa/users/6c5c0983061d423690d951643b92a5b5/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:28.916786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29915a5e-9751-419a-bac7-1c259346bfcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2425c01d8014202b624ce72c24b8baa, user_id=6c5c0983061d423690d951643b92a5b5, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:28.916786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29915a5e-9751-419a-bac7-1c259346bfcb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:28.929807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 955/1939] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:28 2022] PUT /identity/v3/projects/b2425c01d8014202b624ce72c24b8baa/users/6c5c0983061d423690d951643b92a5b5/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:28.930770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29915a5e-9751-419a-bac7-1c259346bfcb admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.013857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29915a5e-9751-419a-bac7-1c259346bfcb admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:29.019042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafc4c4-4500-4c78-841b-82394d7ea96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:29.019672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc4c4-4500-4c78-841b-82394d7ea96c admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.020548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 985/1940] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:28 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:29.021109 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc4c4-4500-4c78-841b-82394d7ea96c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.021404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc4c4-4500-4c78-841b-82394d7ea96c admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.021800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1f1f1-78cc-4955-8877-6889acfa82a7 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.022086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafc4c4-4500-4c78-841b-82394d7ea96c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:29.022398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafc4c4-4500-4c78-841b-82394d7ea96c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:29.024213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafc4c4-4500-4c78-841b-82394d7ea96c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:29.032703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1f1f1-78cc-4955-8877-6889acfa82a7 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:29.038183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 956/1941] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:03:29.039537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753b0bd-e5e5-4769-912e-3df55945f963 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.039824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753b0bd-e5e5-4769-912e-3df55945f963 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.039920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb45cad-4788-4932-84f8-693e3aedc16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:29.040102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753b0bd-e5e5-4769-912e-3df55945f963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.041061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb45cad-4788-4932-84f8-693e3aedc16c admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.042203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb45cad-4788-4932-84f8-693e3aedc16c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.043060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb45cad-4788-4932-84f8-693e3aedc16c admin admin] PATH_INFO: `/v3/projects/dd2e1cfdfc2f4efba65aea6be02e456b/users/e30069ef241647699f41cbd8588c279d/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.049040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb45cad-4788-4932-84f8-693e3aedc16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd2e1cfdfc2f4efba65aea6be02e456b, user_id=e30069ef241647699f41cbd8588c279d, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:29.055768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb45cad-4788-4932-84f8-693e3aedc16c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:29.074689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-9753b0bd-e5e5-4769-912e-3df55945f963 None None] MFA Rules not processed for user `6c5c0983061d423690d951643b92a5b5`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:29.077479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-9753b0bd-e5e5-4769-912e-3df55945f963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:29.078144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 986/1942] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:29 2022] PUT /identity/v3/projects/dd2e1cfdfc2f4efba65aea6be02e456b/users/e30069ef241647699f41cbd8588c279d/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:29.079603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb45cad-4788-4932-84f8-693e3aedc16c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.166797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb45cad-4788-4932-84f8-693e3aedc16c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:29.172104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851abac4-f1d5-49bd-b8af-357a5551f084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:29.172767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851abac4-f1d5-49bd-b8af-357a5551f084 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.173143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851abac4-f1d5-49bd-b8af-357a5551f084 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.173308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851abac4-f1d5-49bd-b8af-357a5551f084 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.173974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851abac4-f1d5-49bd-b8af-357a5551f084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:29.176328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851abac4-f1d5-49bd-b8af-357a5551f084 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:29.187144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 957/1943] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:29.188074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 987/1944] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:29.189228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851abac4-f1d5-49bd-b8af-357a5551f084 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.191099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974477a6-ebfc-4294-a60a-350416ff17bc None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.192009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974477a6-ebfc-4294-a60a-350416ff17bc None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.192774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974477a6-ebfc-4294-a60a-350416ff17bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.207555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851abac4-f1d5-49bd-b8af-357a5551f084 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:29.215039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4bef8-a9b3-4d04-9dde-1ddb8ae855c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:29.216098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4bef8-a9b3-4d04-9dde-1ddb8ae855c5 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.216571 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4bef8-a9b3-4d04-9dde-1ddb8ae855c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.217330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4bef8-a9b3-4d04-9dde-1ddb8ae855c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.218230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4bef8-a9b3-4d04-9dde-1ddb8ae855c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:29.223200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4bef8-a9b3-4d04-9dde-1ddb8ae855c5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:29.254582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-974477a6-ebfc-4294-a60a-350416ff17bc None None] MFA Rules not processed for user `e30069ef241647699f41cbd8588c279d`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:29.256849 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-974477a6-ebfc-4294-a60a-350416ff17bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:29.358478 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 958/1945] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:29.359516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974477a6-ebfc-4294-a60a-350416ff17bc None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.378117 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974477a6-ebfc-4294-a60a-350416ff17bc None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:29.384358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f6a6f-af1a-45c4-b53a-c9f3c166ee5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:29.387640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f6a6f-af1a-45c4-b53a-c9f3c166ee5c admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.390451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f6a6f-af1a-45c4-b53a-c9f3c166ee5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.390451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f6a6f-af1a-45c4-b53a-c9f3c166ee5c admin admin] PATH_INFO: `/v3/projects/2b3d7d28b00b401588c618354651238b/users/e06e1ee3896d4a23ad23bc625b91166b/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.396825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 988/1946] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:29 2022] POST /identity/v3/projects => generated 375 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:29.397894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4bef8-a9b3-4d04-9dde-1ddb8ae855c5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.399625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c5e4bef8-a9b3-4d04-9dde-1ddb8ae855c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:29.437658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f6a6f-af1a-45c4-b53a-c9f3c166ee5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3d7d28b00b401588c618354651238b, user_id=e06e1ee3896d4a23ad23bc625b91166b, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:29.445933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f6a6f-af1a-45c4-b53a-c9f3c166ee5c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:29.479973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4bef8-a9b3-4d04-9dde-1ddb8ae855c5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:29.484070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a0b4c-a3ab-4e8d-bbab-7591ee02c3df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:29.484759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a0b4c-a3ab-4e8d-bbab-7591ee02c3df service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.485043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a0b4c-a3ab-4e8d-bbab-7591ee02c3df service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.485342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a0b4c-a3ab-4e8d-bbab-7591ee02c3df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.543779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 959/1947] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:29 2022] PUT /identity/v3/projects/2b3d7d28b00b401588c618354651238b/users/e06e1ee3896d4a23ad23bc625b91166b/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:29.544853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f6a6f-af1a-45c4-b53a-c9f3c166ee5c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.546935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-fd4f6a6f-af1a-45c4-b53a-c9f3c166ee5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:29.574107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a0b4c-a3ab-4e8d-bbab-7591ee02c3df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:29.575838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a0b4c-a3ab-4e8d-bbab-7591ee02c3df service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:29.593233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 989/1948] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:29.594480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a0b4c-a3ab-4e8d-bbab-7591ee02c3df service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.636375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f6a6f-af1a-45c4-b53a-c9f3c166ee5c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:29.650944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37370ff1-93ea-4718-87fe-ab209582aeb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:29.651578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37370ff1-93ea-4718-87fe-ab209582aeb6 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.651859 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37370ff1-93ea-4718-87fe-ab209582aeb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.652129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37370ff1-93ea-4718-87fe-ab209582aeb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.722388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:29.723248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a0b4c-a3ab-4e8d-bbab-7591ee02c3df service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:29.746805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37370ff1-93ea-4718-87fe-ab209582aeb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:29.752903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37370ff1-93ea-4718-87fe-ab209582aeb6 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:29.765207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 960/1949] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:29.766092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37370ff1-93ea-4718-87fe-ab209582aeb6 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.813269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78694f-81ed-4006-b9ed-5ab5b02a4359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:29.814067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78694f-81ed-4006-b9ed-5ab5b02a4359 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.816073 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78694f-81ed-4006-b9ed-5ab5b02a4359 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.816714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78694f-81ed-4006-b9ed-5ab5b02a4359 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.818853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78694f-81ed-4006-b9ed-5ab5b02a4359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:29.821198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78694f-81ed-4006-b9ed-5ab5b02a4359 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:29.865623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37370ff1-93ea-4718-87fe-ab209582aeb6 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:29.873932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703e07c-0511-47e5-9c75-96ec9b715356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:29.874785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e07c-0511-47e5-9c75-96ec9b715356 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.875068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e07c-0511-47e5-9c75-96ec9b715356 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.875356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e07c-0511-47e5-9c75-96ec9b715356 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.876017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703e07c-0511-47e5-9c75-96ec9b715356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:29.878547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703e07c-0511-47e5-9c75-96ec9b715356 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:29.898741 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 961/1950] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:29.902897 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703e07c-0511-47e5-9c75-96ec9b715356 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.916922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703e07c-0511-47e5-9c75-96ec9b715356 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:29.933768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa3225-a437-4f7b-9c48-2106cf54e7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:29.935968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa3225-a437-4f7b-9c48-2106cf54e7ff admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.935968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa3225-a437-4f7b-9c48-2106cf54e7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.935968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa3225-a437-4f7b-9c48-2106cf54e7ff admin admin] PATH_INFO: `/v3/projects/2b3d7d28b00b401588c618354651238b/users/e06e1ee3896d4a23ad23bc625b91166b/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:29.939090 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa3225-a437-4f7b-9c48-2106cf54e7ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3d7d28b00b401588c618354651238b, user_id=e06e1ee3896d4a23ad23bc625b91166b, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:29.943900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa3225-a437-4f7b-9c48-2106cf54e7ff admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:29.966082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 962/1951] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:29 2022] PUT /identity/v3/projects/2b3d7d28b00b401588c618354651238b/users/e06e1ee3896d4a23ad23bc625b91166b/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:29.974357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 990/1952] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:29 2022] POST /identity/v3/users => generated 288 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:29.984982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa3225-a437-4f7b-9c48-2106cf54e7ff admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:29.990536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f98fd-1807-4b79-9d4f-e016e4637ca7 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:29.991328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f98fd-1807-4b79-9d4f-e016e4637ca7 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:29.991814 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f98fd-1807-4b79-9d4f-e016e4637ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:30.026991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-fc4f98fd-1807-4b79-9d4f-e016e4637ca7 None None] MFA Rules not processed for user `e06e1ee3896d4a23ad23bc625b91166b`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:30.029195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fc4f98fd-1807-4b79-9d4f-e016e4637ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:30.082441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa3225-a437-4f7b-9c48-2106cf54e7ff admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:30.087651 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51afcc7-3fa4-4bd8-a1d0-5bc20b86c146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:30.088306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51afcc7-3fa4-4bd8-a1d0-5bc20b86c146 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:30.088579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51afcc7-3fa4-4bd8-a1d0-5bc20b86c146 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:30.088843 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51afcc7-3fa4-4bd8-a1d0-5bc20b86c146 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:30.089502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51afcc7-3fa4-4bd8-a1d0-5bc20b86c146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:30.091120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51afcc7-3fa4-4bd8-a1d0-5bc20b86c146 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:30.100374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:30.103924 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 963/1953] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:30.105129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51afcc7-3fa4-4bd8-a1d0-5bc20b86c146 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:30.142316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 991/1954] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:30.142316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f98fd-1807-4b79-9d4f-e016e4637ca7 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:30.156733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f98fd-1807-4b79-9d4f-e016e4637ca7 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:30.165883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad97f16-6c09-4103-a78a-28dbeb429336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:30.167103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97f16-6c09-4103-a78a-28dbeb429336 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:30.167951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97f16-6c09-4103-a78a-28dbeb429336 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:30.168254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97f16-6c09-4103-a78a-28dbeb429336 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:30.169432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad97f16-6c09-4103-a78a-28dbeb429336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:30.172591 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad97f16-6c09-4103-a78a-28dbeb429336 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:30.185032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51afcc7-3fa4-4bd8-a1d0-5bc20b86c146 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:30.189412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304746-456e-456b-a4f2-aef55e748c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:30.191168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304746-456e-456b-a4f2-aef55e748c74 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:30.191168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304746-456e-456b-a4f2-aef55e748c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:30.191463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304746-456e-456b-a4f2-aef55e748c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:30.193283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ce304746-456e-456b-a4f2-aef55e748c74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:30.200320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 992/1955] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:30.208730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad97f16-6c09-4103-a78a-28dbeb429336 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:30.225028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad97f16-6c09-4103-a78a-28dbeb429336 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:30.230091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fb748-e313-4a9c-b6c4-f39b8d5c640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:30.230785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fb748-e313-4a9c-b6c4-f39b8d5c640a admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:30.231067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fb748-e313-4a9c-b6c4-f39b8d5c640a admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:30.231349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fb748-e313-4a9c-b6c4-f39b8d5c640a admin admin] PATH_INFO: `/v3/projects/1adf43ebb9b84aec9beb89d36fd48eb3/users/d2cbf52a4f714ad292b5088ab7c4d52d/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:30.248309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fb748-e313-4a9c-b6c4-f39b8d5c640a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adf43ebb9b84aec9beb89d36fd48eb3, user_id=d2cbf52a4f714ad292b5088ab7c4d52d, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:30.251064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fb748-e313-4a9c-b6c4-f39b8d5c640a admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:30.310096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce304746-456e-456b-a4f2-aef55e748c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:30.312611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce304746-456e-456b-a4f2-aef55e748c74 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:30.330638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 964/1956] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:30.332285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304746-456e-456b-a4f2-aef55e748c74 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:30.346194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304746-456e-456b-a4f2-aef55e748c74 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:30.355607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 993/1957] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:30 2022] PUT /identity/v3/projects/1adf43ebb9b84aec9beb89d36fd48eb3/users/d2cbf52a4f714ad292b5088ab7c4d52d/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:30.372387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50305c1-6bc3-413b-a078-027a0a6610c4 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:30.372856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50305c1-6bc3-413b-a078-027a0a6610c4 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:30.373707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50305c1-6bc3-413b-a078-027a0a6610c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:30.424988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f50305c1-6bc3-413b-a078-027a0a6610c4 None None] MFA Rules not processed for user `d2cbf52a4f714ad292b5088ab7c4d52d`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:30.427505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f50305c1-6bc3-413b-a078-027a0a6610c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:30.440865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846dc75-37ca-4f36-8a6c-e1f4f614efbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:30.441382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846dc75-37ca-4f36-8a6c-e1f4f614efbc service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:30.441681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846dc75-37ca-4f36-8a6c-e1f4f614efbc service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:30.441938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846dc75-37ca-4f36-8a6c-e1f4f614efbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:30.539314 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 994/1958] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:30.550577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846dc75-37ca-4f36-8a6c-e1f4f614efbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:30.553023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846dc75-37ca-4f36-8a6c-e1f4f614efbc service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:30.572233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 965/1959] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:30.582848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:30.674047 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:30.771961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50305c1-6bc3-413b-a078-027a0a6610c4 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:30.784260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50305c1-6bc3-413b-a078-027a0a6610c4 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:30.792265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea25f8-238a-4795-8001-17462f2d5129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:30.793209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea25f8-238a-4795-8001-17462f2d5129 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:30.793588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea25f8-238a-4795-8001-17462f2d5129 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:30.794014 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea25f8-238a-4795-8001-17462f2d5129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:30.807932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea25f8-238a-4795-8001-17462f2d5129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:30.810486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea25f8-238a-4795-8001-17462f2d5129 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:30.820776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 995/1960] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:31.709286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:31.982778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:32.064810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846dc75-37ca-4f36-8a6c-e1f4f614efbc service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:32.096365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846dc75-37ca-4f36-8a6c-e1f4f614efbc service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:32.117448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b222c0f-ebe5-446b-bfe5-e869618c3aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:32.118172 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b222c0f-ebe5-446b-bfe5-e869618c3aab service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:32.118596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b222c0f-ebe5-446b-bfe5-e869618c3aab service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:32.119056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b222c0f-ebe5-446b-bfe5-e869618c3aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:32.122045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0b222c0f-ebe5-446b-bfe5-e869618c3aab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:32.262559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b222c0f-ebe5-446b-bfe5-e869618c3aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:32.266181 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b222c0f-ebe5-446b-bfe5-e869618c3aab service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:32.289189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 966/1961] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:32.988389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:33.056135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea25f8-238a-4795-8001-17462f2d5129 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:33.178631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea25f8-238a-4795-8001-17462f2d5129 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:33.184741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:33.192278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:33.193182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:33.193588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:33.194058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] PATH_INFO: `/v3/users/5ff987ea8e0a485b9f88fcd365269d33` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:33.195614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff987ea8e0a485b9f88fcd365269d33)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:33.197824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:33.531696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff987ea8e0a485b9f88fcd365269d33'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:33.533675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff987ea8e0a485b9f88fcd365269d33'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:33.550815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff987ea8e0a485b9f88fcd365269d33'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:33.552468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff987ea8e0a485b9f88fcd365269d33'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:33.678307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff987ea8e0a485b9f88fcd365269d33'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:33.711217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 996/1962] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:33 2022] DELETE /identity/v3/users/5ff987ea8e0a485b9f88fcd365269d33 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:33.716657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2dbf9-0203-44ab-8fac-7f67bfa4fbf5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:33.717291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2dbf9-0203-44ab-8fac-7f67bfa4fbf5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:33.717708 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2dbf9-0203-44ab-8fac-7f67bfa4fbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:33.748807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-1ab2dbf9-0203-44ab-8fac-7f67bfa4fbf5 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:33.750456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1ab2dbf9-0203-44ab-8fac-7f67bfa4fbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:33.866452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 967/1963] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:33.887544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:33.893053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:33.960320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:34.014914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c81d69-f361-48c1-8ffb-ec16ea4e8b9e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:34.023332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9645798-df7c-4ed5-afa3-363919eabf5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:34.024383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9645798-df7c-4ed5-afa3-363919eabf5f service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:34.024865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9645798-df7c-4ed5-afa3-363919eabf5f service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:34.025338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9645798-df7c-4ed5-afa3-363919eabf5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:34.043793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9645798-df7c-4ed5-afa3-363919eabf5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:34.046504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9645798-df7c-4ed5-afa3-363919eabf5f service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:34.064683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 997/1964] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:34.098430 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:34.454611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2dbf9-0203-44ab-8fac-7f67bfa4fbf5 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:34.457632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1ab2dbf9-0203-44ab-8fac-7f67bfa4fbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:34.485886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2dbf9-0203-44ab-8fac-7f67bfa4fbf5 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:34.495039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:34.496511 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:34.496984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:34.497646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] PATH_INFO: `/v3/users/49a32a1453f34563934bec7ffe9f4346` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:34.500091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a32a1453f34563934bec7ffe9f4346)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:34.504832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:34.829056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a32a1453f34563934bec7ffe9f4346'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:34.840956 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a32a1453f34563934bec7ffe9f4346'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:34.843084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a32a1453f34563934bec7ffe9f4346'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:34.941112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a32a1453f34563934bec7ffe9f4346'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:34.964916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a32a1453f34563934bec7ffe9f4346'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:34.972386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 968/1965] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:34 2022] DELETE /identity/v3/users/49a32a1453f34563934bec7ffe9f4346 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:34.980867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90145a-fbaa-46f2-b3c0-138bee3cb77d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:34.981388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90145a-fbaa-46f2-b3c0-138bee3cb77d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:34.981895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90145a-fbaa-46f2-b3c0-138bee3cb77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:35.010987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2f90145a-fbaa-46f2-b3c0-138bee3cb77d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:35.014554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2f90145a-fbaa-46f2-b3c0-138bee3cb77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:35.054899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:35.058098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:35.168671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 998/1966] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:35.185167 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90145a-fbaa-46f2-b3c0-138bee3cb77d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:35.187118 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2f90145a-fbaa-46f2-b3c0-138bee3cb77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:35.243240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36168a1-b49f-43dc-9031-d1498987c9a9 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:35.252836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:35.254774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:35.255281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:35.255791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] PATH_INFO: `/v3/projects/24332f4006014757a7eda8bfb5d7ad8d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:35.258362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24332f4006014757a7eda8bfb5d7ad8d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:35.262899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:35.280264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90145a-fbaa-46f2-b3c0-138bee3cb77d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:35.293059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eac18a-be08-488b-bde1-6b8ea1786d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:35.294150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eac18a-be08-488b-bde1-6b8ea1786d56 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:35.295774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eac18a-be08-488b-bde1-6b8ea1786d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:35.296105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eac18a-be08-488b-bde1-6b8ea1786d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:35.316257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eac18a-be08-488b-bde1-6b8ea1786d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:35.318868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eac18a-be08-488b-bde1-6b8ea1786d56 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:35.329528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:35.330911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 999/1967] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:35.485809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24332f4006014757a7eda8bfb5d7ad8d', 'request_id': 'req-0e4b472c-bfc8-4429-8d62-75ca94070905'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:35.497187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24332f4006014757a7eda8bfb5d7ad8d', 'request_id': 'req-0e4b472c-bfc8-4429-8d62-75ca94070905'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:35.508615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24332f4006014757a7eda8bfb5d7ad8d', 'request_id': 'req-0e4b472c-bfc8-4429-8d62-75ca94070905'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:35.624927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] The token cache is being invalidate because project 24332f4006014757a7eda8bfb5d7ad8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:35.625583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:35.627169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:35.634070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 969/1968] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:35 2022] DELETE /identity/v3/projects/24332f4006014757a7eda8bfb5d7ad8d => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:35.636280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eac18a-be08-488b-bde1-6b8ea1786d56 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:35.649464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eac18a-be08-488b-bde1-6b8ea1786d56 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:35.745638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:35.746297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:35.746597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:35.746847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] PATH_INFO: `/v3/users/e30069ef241647699f41cbd8588c279d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:35.748122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e30069ef241647699f41cbd8588c279d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:35.750750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:35.766536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:36.074593 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e30069ef241647699f41cbd8588c279d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:36.075820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e30069ef241647699f41cbd8588c279d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:36.092930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e30069ef241647699f41cbd8588c279d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:36.094450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e30069ef241647699f41cbd8588c279d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:36.169519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:36.173107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:36.198755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e30069ef241647699f41cbd8588c279d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:36.204953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b472c-bfc8-4429-8d62-75ca94070905 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:36.215165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:36.216669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:36.217381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:36.218200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] PATH_INFO: `/v3/projects/658f63649667484cbd7a11de765e5bd3` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:36.222328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] RBAC: Authorizing `identity:delete_project(project_id=658f63649667484cbd7a11de765e5bd3)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:36.227638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:36.256121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1000/1969] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:35 2022] DELETE /identity/v3/users/e30069ef241647699f41cbd8588c279d => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:36.456752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658f63649667484cbd7a11de765e5bd3', 'request_id': 'req-06575bf3-cf9b-479f-8a87-1fcc3275397d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:36.465670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658f63649667484cbd7a11de765e5bd3', 'request_id': 'req-06575bf3-cf9b-479f-8a87-1fcc3275397d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:36.475080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658f63649667484cbd7a11de765e5bd3', 'request_id': 'req-06575bf3-cf9b-479f-8a87-1fcc3275397d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:36.646256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] The token cache is being invalidate because project 658f63649667484cbd7a11de765e5bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:36.649381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:36.652011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:36.666429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 970/1970] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:36 2022] DELETE /identity/v3/projects/658f63649667484cbd7a11de765e5bd3 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:36.667639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:36.669963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:36.879018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57378493-134c-4236-82a8-e8dbca509cdc admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:36.906792 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:36.981587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:36.982347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:36.982761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:36.983165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] PATH_INFO: `/v3/users/e06e1ee3896d4a23ad23bc625b91166b` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:36.984545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e06e1ee3896d4a23ad23bc625b91166b)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:36.986617 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:37.020798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:37.023527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:37.078904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06575bf3-cf9b-479f-8a87-1fcc3275397d admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:37.087635 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:37.088996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:37.089488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:37.089856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] PATH_INFO: `/v3/projects/dd2e1cfdfc2f4efba65aea6be02e456b` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:37.092153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd2e1cfdfc2f4efba65aea6be02e456b)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:37.095925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:37.306877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e06e1ee3896d4a23ad23bc625b91166b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:37.308055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e06e1ee3896d4a23ad23bc625b91166b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:37.314767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd2e1cfdfc2f4efba65aea6be02e456b', 'request_id': 'req-107acf26-fa15-4d04-80be-c1445101a5ce'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:37.327387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e06e1ee3896d4a23ad23bc625b91166b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:37.331930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e06e1ee3896d4a23ad23bc625b91166b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:37.332292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd2e1cfdfc2f4efba65aea6be02e456b', 'request_id': 'req-107acf26-fa15-4d04-80be-c1445101a5ce'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:37.344810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd2e1cfdfc2f4efba65aea6be02e456b', 'request_id': 'req-107acf26-fa15-4d04-80be-c1445101a5ce'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:37.433045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7e85e6f5-6c36-4540-ac24-e7efdafe4dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e06e1ee3896d4a23ad23bc625b91166b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:37.462095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:37.472756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1001/1971] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:36 2022] DELETE /identity/v3/users/e06e1ee3896d4a23ad23bc625b91166b => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:37.533107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] The token cache is being invalidate because project dd2e1cfdfc2f4efba65aea6be02e456b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:37.534760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:37.536055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-107acf26-fa15-4d04-80be-c1445101a5ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:37.543283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 971/1972] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:37 2022] DELETE /identity/v3/projects/dd2e1cfdfc2f4efba65aea6be02e456b => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:37.548275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ebfd8-1141-4311-80dc-3c1044a6f27c None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:37.551141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ebfd8-1141-4311-80dc-3c1044a6f27c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:37.553139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ebfd8-1141-4311-80dc-3c1044a6f27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:37.639828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-572ebfd8-1141-4311-80dc-3c1044a6f27c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:37.641596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-572ebfd8-1141-4311-80dc-3c1044a6f27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:37.743205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1002/1973] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:37.752320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a6d13-7765-4699-9056-2f81590d0639 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:37.753745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a6d13-7765-4699-9056-2f81590d0639 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:37.754266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a6d13-7765-4699-9056-2f81590d0639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:37.778592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-c04a6d13-7765-4699-9056-2f81590d0639 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:37.780575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c04a6d13-7765-4699-9056-2f81590d0639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:37.794470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 972/1974] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:37.795420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ebfd8-1141-4311-80dc-3c1044a6f27c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:37.808468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ebfd8-1141-4311-80dc-3c1044a6f27c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:37.816427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87361d8-34fe-4800-ba1c-123f58a8a364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:37.817068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87361d8-34fe-4800-ba1c-123f58a8a364 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:37.817356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87361d8-34fe-4800-ba1c-123f58a8a364 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:37.817630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87361d8-34fe-4800-ba1c-123f58a8a364 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:37.818307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87361d8-34fe-4800-ba1c-123f58a8a364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:37.818616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87361d8-34fe-4800-ba1c-123f58a8a364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:37.820340 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87361d8-34fe-4800-ba1c-123f58a8a364 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:37.843084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1003/1975] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:03:37.854018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a6d13-7765-4699-9056-2f81590d0639 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:37.866258 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a6d13-7765-4699-9056-2f81590d0639 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:37.871898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8e3eb-39d5-437f-986f-ee6a58244954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:37.872660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8e3eb-39d5-437f-986f-ee6a58244954 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:37.873073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8e3eb-39d5-437f-986f-ee6a58244954 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:37.873442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8e3eb-39d5-437f-986f-ee6a58244954 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:37.874329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8e3eb-39d5-437f-986f-ee6a58244954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:37.876706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8e3eb-39d5-437f-986f-ee6a58244954 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:38.034733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:38.056125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 973/1976] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:37 2022] POST /identity/v3/projects => generated 371 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:03:38.064707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87361d8-34fe-4800-ba1c-123f58a8a364 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:38.090335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87361d8-34fe-4800-ba1c-123f58a8a364 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:38.235246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:38.317916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70df74-66cc-45ce-8b27-e5a5f65201a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:38.319154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70df74-66cc-45ce-8b27-e5a5f65201a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:38.319748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70df74-66cc-45ce-8b27-e5a5f65201a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:38.320376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70df74-66cc-45ce-8b27-e5a5f65201a4 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:38.321921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70df74-66cc-45ce-8b27-e5a5f65201a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:38.325875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70df74-66cc-45ce-8b27-e5a5f65201a4 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:38.405810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8e3eb-39d5-437f-986f-ee6a58244954 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:38.408474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-18b8e3eb-39d5-437f-986f-ee6a58244954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:38.447786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8e3eb-39d5-437f-986f-ee6a58244954 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:38.455054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:38.456675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:38.457122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:38.458846 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] PATH_INFO: `/v3/projects/2b3d7d28b00b401588c618354651238b` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:38.459944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3d7d28b00b401588c618354651238b)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:38.463929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:38.488679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1004/1977] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:38 2022] POST /identity/v3/users => generated 286 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:38.493295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70df74-66cc-45ce-8b27-e5a5f65201a4 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:38.511174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70df74-66cc-45ce-8b27-e5a5f65201a4 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:38.521123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036b17b-37dc-414a-9213-38ad91c066eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:38.521864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036b17b-37dc-414a-9213-38ad91c066eb admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:38.522220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036b17b-37dc-414a-9213-38ad91c066eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:38.527699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036b17b-37dc-414a-9213-38ad91c066eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:38.528691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036b17b-37dc-414a-9213-38ad91c066eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:38.531491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036b17b-37dc-414a-9213-38ad91c066eb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:38.549214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1005/1978] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:38.556078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036b17b-37dc-414a-9213-38ad91c066eb admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:38.584929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036b17b-37dc-414a-9213-38ad91c066eb admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:38.598867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f057f6-0d07-4c65-8090-172ca2c3a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:38.600428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f057f6-0d07-4c65-8090-172ca2c3a9cc admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:38.601166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f057f6-0d07-4c65-8090-172ca2c3a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:38.601750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f057f6-0d07-4c65-8090-172ca2c3a9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:38.603189 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f057f6-0d07-4c65-8090-172ca2c3a9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:38.606739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f057f6-0d07-4c65-8090-172ca2c3a9cc admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:38.625652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1006/1979] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:38.631272 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f057f6-0d07-4c65-8090-172ca2c3a9cc admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:38.651305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f057f6-0d07-4c65-8090-172ca2c3a9cc admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:38.674788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7976591-a21f-4f66-92ab-bdcb18af23cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:38.674788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7976591-a21f-4f66-92ab-bdcb18af23cd admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:38.674788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7976591-a21f-4f66-92ab-bdcb18af23cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:38.674788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7976591-a21f-4f66-92ab-bdcb18af23cd admin admin] PATH_INFO: `/v3/projects/0a766cf899434ae98684ffa06f412e2c/users/70d76a9080964bc2899ab57e10818e3d/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:38.693409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3d7d28b00b401588c618354651238b', 'request_id': 'req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:38.715039 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3d7d28b00b401588c618354651238b', 'request_id': 'req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:38.719421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7976591-a21f-4f66-92ab-bdcb18af23cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a766cf899434ae98684ffa06f412e2c, user_id=70d76a9080964bc2899ab57e10818e3d, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:38.724402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3d7d28b00b401588c618354651238b', 'request_id': 'req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:38.726251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7976591-a21f-4f66-92ab-bdcb18af23cd admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:38.852467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] The token cache is being invalidate because project 2b3d7d28b00b401588c618354651238b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:38.854009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:38.858282 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8e1473a1-c087-4c6c-88f6-69ced33ac4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:38.860235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1007/1980] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:38 2022] PUT /identity/v3/projects/0a766cf899434ae98684ffa06f412e2c/users/70d76a9080964bc2899ab57e10818e3d/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:38.868948 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2162fe-6bd7-4a2a-9484-3b41878d2e99 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:38.869832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 974/1981] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:38 2022] DELETE /identity/v3/projects/2b3d7d28b00b401588c618354651238b => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:38.870373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2162fe-6bd7-4a2a-9484-3b41878d2e99 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:38.871179 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2162fe-6bd7-4a2a-9484-3b41878d2e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:38.872208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db8b82-c3be-45e5-a561-4b280b66225e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:38.872744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db8b82-c3be-45e5-a561-4b280b66225e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:38.873101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db8b82-c3be-45e5-a561-4b280b66225e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:38.915207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d6db8b82-c3be-45e5-a561-4b280b66225e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:38.917571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d6db8b82-c3be-45e5-a561-4b280b66225e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:38.918141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-7e2162fe-6bd7-4a2a-9484-3b41878d2e99 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:38.919752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7e2162fe-6bd7-4a2a-9484-3b41878d2e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:39.056388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 975/1982] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:39.064191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3224860-c72a-4116-b647-6ab144ea52ec None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:39.064502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3224860-c72a-4116-b647-6ab144ea52ec None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:39.064782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3224860-c72a-4116-b647-6ab144ea52ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:39.092324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1008/1983] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:39.092611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f3224860-c72a-4116-b647-6ab144ea52ec None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:39.093037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f3224860-c72a-4116-b647-6ab144ea52ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:39.104026 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 976/1984] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:39.105328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2162fe-6bd7-4a2a-9484-3b41878d2e99 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:39.107137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7e2162fe-6bd7-4a2a-9484-3b41878d2e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:39.109935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3224860-c72a-4116-b647-6ab144ea52ec None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:39.138559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3224860-c72a-4116-b647-6ab144ea52ec None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:39.148587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f0b69-6f7c-4903-a750-1512bd2ff103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:39.149790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f0b69-6f7c-4903-a750-1512bd2ff103 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:39.150399 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f0b69-6f7c-4903-a750-1512bd2ff103 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:39.151691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f0b69-6f7c-4903-a750-1512bd2ff103 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:39.153006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f0b69-6f7c-4903-a750-1512bd2ff103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:39.153943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f0b69-6f7c-4903-a750-1512bd2ff103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:39.157991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f0b69-6f7c-4903-a750-1512bd2ff103 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:39.188574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 977/1985] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:03:39.205285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f0b69-6f7c-4903-a750-1512bd2ff103 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:39.211269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f0b69-6f7c-4903-a750-1512bd2ff103 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:39.217954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a3759-c307-4a2b-b513-fbd6f8df65b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:39.218697 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a3759-c307-4a2b-b513-fbd6f8df65b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:39.219117 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a3759-c307-4a2b-b513-fbd6f8df65b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:39.219636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a3759-c307-4a2b-b513-fbd6f8df65b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:39.220572 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a3759-c307-4a2b-b513-fbd6f8df65b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:39.223681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a3759-c307-4a2b-b513-fbd6f8df65b6 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:39.236292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2162fe-6bd7-4a2a-9484-3b41878d2e99 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:39.244205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d0218-aa23-4acf-b831-6ceb4a8a6f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:39.244754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d0218-aa23-4acf-b831-6ceb4a8a6f8e service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:39.245037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d0218-aa23-4acf-b831-6ceb4a8a6f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:39.245313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d0218-aa23-4acf-b831-6ceb4a8a6f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:39.261522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d0218-aa23-4acf-b831-6ceb4a8a6f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:39.263069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d0218-aa23-4acf-b831-6ceb4a8a6f8e service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:39.277928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1009/1986] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:39.450175 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:39.503545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 978/1987] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:39 2022] POST /identity/v3/projects => generated 371 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:03:39.510523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d0218-aa23-4acf-b831-6ceb4a8a6f8e service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:39.530887 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:39.538359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d0218-aa23-4acf-b831-6ceb4a8a6f8e service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:39.652643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32fecc-9602-4239-9038-72e82133b1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:39.653318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32fecc-9602-4239-9038-72e82133b1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:39.653649 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32fecc-9602-4239-9038-72e82133b1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:39.656960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32fecc-9602-4239-9038-72e82133b1d5 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:39.656960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32fecc-9602-4239-9038-72e82133b1d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:39.657295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32fecc-9602-4239-9038-72e82133b1d5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:39.812151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1010/1988] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:39 2022] POST /identity/v3/users => generated 286 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:03:39.818996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a3759-c307-4a2b-b513-fbd6f8df65b6 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:39.842559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a3759-c307-4a2b-b513-fbd6f8df65b6 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:39.852711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402edc02-93ba-4688-8cda-a898fb7ce714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:39.855380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402edc02-93ba-4688-8cda-a898fb7ce714 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:39.856209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402edc02-93ba-4688-8cda-a898fb7ce714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:39.856841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402edc02-93ba-4688-8cda-a898fb7ce714 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:39.859220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402edc02-93ba-4688-8cda-a898fb7ce714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:39.863424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402edc02-93ba-4688-8cda-a898fb7ce714 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:39.875461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:39.902178 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32fecc-9602-4239-9038-72e82133b1d5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:39.911539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 979/1989] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:39.925120 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32fecc-9602-4239-9038-72e82133b1d5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:39.935719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5ea5a-be5c-4868-9c87-4dc54119b58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:39.937192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ea5a-be5c-4868-9c87-4dc54119b58d admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:39.938069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ea5a-be5c-4868-9c87-4dc54119b58d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:39.939069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ea5a-be5c-4868-9c87-4dc54119b58d admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:39.941253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5ea5a-be5c-4868-9c87-4dc54119b58d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:39.945410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5ea5a-be5c-4868-9c87-4dc54119b58d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:39.976669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402edc02-93ba-4688-8cda-a898fb7ce714 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:39.986032 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1011/1990] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:39.997338 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402edc02-93ba-4688-8cda-a898fb7ce714 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:40.003672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf5cf9-82da-4bb4-a0a0-ca8080c54481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:40.004961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf5cf9-82da-4bb4-a0a0-ca8080c54481 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:40.004961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf5cf9-82da-4bb4-a0a0-ca8080c54481 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:40.005266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf5cf9-82da-4bb4-a0a0-ca8080c54481 admin admin] PATH_INFO: `/v3/projects/4ff033e7c38e4be9a9c3ed8f0bca95dc/users/ace43cc48aec417199cbc44a7c04a0c2/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:40.023540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf5cf9-82da-4bb4-a0a0-ca8080c54481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff033e7c38e4be9a9c3ed8f0bca95dc, user_id=ace43cc48aec417199cbc44a7c04a0c2, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:40.026458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf5cf9-82da-4bb4-a0a0-ca8080c54481 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:40.109531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:40.196239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 980/1991] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:39 2022] PUT /identity/v3/projects/4ff033e7c38e4be9a9c3ed8f0bca95dc/users/ace43cc48aec417199cbc44a7c04a0c2/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:40.215079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eeab38-be75-490d-af03-da080d402c8f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:40.215548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eeab38-be75-490d-af03-da080d402c8f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:40.216043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eeab38-be75-490d-af03-da080d402c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:40.293767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-18eeab38-be75-490d-af03-da080d402c8f None None] MFA Rules not processed for user `ace43cc48aec417199cbc44a7c04a0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:40.296849 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-18eeab38-be75-490d-af03-da080d402c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:40.440080 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1012/1992] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:40.503994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf5cf9-82da-4bb4-a0a0-ca8080c54481 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:40.509422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0bdf5cf9-82da-4bb4-a0a0-ca8080c54481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:40.589805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf5cf9-82da-4bb4-a0a0-ca8080c54481 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:40.594043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac0020-95d5-47d0-ade7-4f5116787e62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:40.594822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac0020-95d5-47d0-ade7-4f5116787e62 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:40.595125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac0020-95d5-47d0-ade7-4f5116787e62 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:40.595546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac0020-95d5-47d0-ade7-4f5116787e62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:40.606370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac0020-95d5-47d0-ade7-4f5116787e62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:40.608373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac0020-95d5-47d0-ade7-4f5116787e62 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:40.621483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 981/1993] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:40.648058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eeab38-be75-490d-af03-da080d402c8f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:40.746542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:40.751948 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eeab38-be75-490d-af03-da080d402c8f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:40.760321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:40.760999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:40.761359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:40.761707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] PATH_INFO: `/v3/users/ace43cc48aec417199cbc44a7c04a0c2` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:40.762941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace43cc48aec417199cbc44a7c04a0c2)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:40.764983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:41.189601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace43cc48aec417199cbc44a7c04a0c2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:41.191268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace43cc48aec417199cbc44a7c04a0c2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:41.204965 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace43cc48aec417199cbc44a7c04a0c2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:41.206135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace43cc48aec417199cbc44a7c04a0c2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:41.321137 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac0020-95d5-47d0-ade7-4f5116787e62 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:41.322809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-18ac0020-95d5-47d0-ade7-4f5116787e62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:41.338038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac0020-95d5-47d0-ade7-4f5116787e62 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:41.344133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:41.345453 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:41.345969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:41.346482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] PATH_INFO: `/v3/users/f2929afb7adb4f78ad18d7134740466c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:41.348812 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2929afb7adb4f78ad18d7134740466c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:41.352662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:41.397611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ff982c81-3b0c-4dd8-9744-fa810d971e75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace43cc48aec417199cbc44a7c04a0c2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:41.417584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1013/1994] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:40 2022] DELETE /identity/v3/users/ace43cc48aec417199cbc44a7c04a0c2 => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:41.422425 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5daeee-4b14-49e0-950b-15dae65a7c86 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:41.422711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5daeee-4b14-49e0-950b-15dae65a7c86 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:41.423019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5daeee-4b14-49e0-950b-15dae65a7c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:41.445388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-da5daeee-4b14-49e0-950b-15dae65a7c86 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:41.445826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:41.446664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-da5daeee-4b14-49e0-950b-15dae65a7c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:41.545618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1014/1995] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:41.560945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5daeee-4b14-49e0-950b-15dae65a7c86 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:41.562980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-da5daeee-4b14-49e0-950b-15dae65a7c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:41.667937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5daeee-4b14-49e0-950b-15dae65a7c86 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:41.690997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da64f21-62d4-4af4-8186-eb938cbdb405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:41.691730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da64f21-62d4-4af4-8186-eb938cbdb405 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:41.692065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da64f21-62d4-4af4-8186-eb938cbdb405 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:41.692522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da64f21-62d4-4af4-8186-eb938cbdb405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:41.703440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da64f21-62d4-4af4-8186-eb938cbdb405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:41.705068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da64f21-62d4-4af4-8186-eb938cbdb405 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:41.716751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1015/1996] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:03:41.718227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82a104-562b-4ebb-a539-22d5620133f6 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:41.718584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82a104-562b-4ebb-a539-22d5620133f6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:41.718927 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82a104-562b-4ebb-a539-22d5620133f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:41.761051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2929afb7adb4f78ad18d7134740466c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:41.789966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2929afb7adb4f78ad18d7134740466c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:41.791955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2929afb7adb4f78ad18d7134740466c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:41.825242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-3e82a104-562b-4ebb-a539-22d5620133f6 None None] MFA Rules not processed for user `70d76a9080964bc2899ab57e10818e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:41.827520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3e82a104-562b-4ebb-a539-22d5620133f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:41.917414 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2929afb7adb4f78ad18d7134740466c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:41.925598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1016/1997] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:41.930378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82a104-562b-4ebb-a539-22d5620133f6 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:41.931917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3e82a104-562b-4ebb-a539-22d5620133f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:41.933989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-35d8a1c6-ed9a-43c9-9952-0853085145fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2929afb7adb4f78ad18d7134740466c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:41.940647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 982/1998] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:41 2022] DELETE /identity/v3/users/f2929afb7adb4f78ad18d7134740466c => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:41.946769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9936be-1991-447c-b5ca-d9fa7fd247e5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:41.947054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9936be-1991-447c-b5ca-d9fa7fd247e5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:41.947330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9936be-1991-447c-b5ca-d9fa7fd247e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:41.950690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82a104-562b-4ebb-a539-22d5620133f6 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:41.976533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-0c9936be-1991-447c-b5ca-d9fa7fd247e5 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:41.977823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0c9936be-1991-447c-b5ca-d9fa7fd247e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:42.051193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 983/1999] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:42.061139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc1564-b2df-478e-b351-3f9c524a8331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:42.062903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc1564-b2df-478e-b351-3f9c524a8331 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:42.063627 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc1564-b2df-478e-b351-3f9c524a8331 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:42.064018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc1564-b2df-478e-b351-3f9c524a8331 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:42.066546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9936be-1991-447c-b5ca-d9fa7fd247e5 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:42.068495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc1564-b2df-478e-b351-3f9c524a8331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:42.069328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0c9936be-1991-447c-b5ca-d9fa7fd247e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:42.072460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc1564-b2df-478e-b351-3f9c524a8331 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:42.176760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9936be-1991-447c-b5ca-d9fa7fd247e5 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:42.181289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b27bb-8d32-4d12-b6ac-e53f0bd33bdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:42.182008 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b27bb-8d32-4d12-b6ac-e53f0bd33bdc service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:42.182356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b27bb-8d32-4d12-b6ac-e53f0bd33bdc service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:42.182697 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b27bb-8d32-4d12-b6ac-e53f0bd33bdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:42.194568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b27bb-8d32-4d12-b6ac-e53f0bd33bdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:42.196144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b27bb-8d32-4d12-b6ac-e53f0bd33bdc service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:42.210311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 984/2000] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:03:42.387882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1017/2001] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:41 2022] POST /identity/v3/projects => generated 373 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:03:42.392576 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b27bb-8d32-4d12-b6ac-e53f0bd33bdc service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:42.403759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b27bb-8d32-4d12-b6ac-e53f0bd33bdc service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:42.455294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:42.471149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bfe45-0bbe-48ed-90b5-0b76f73846ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:42.471855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bfe45-0bbe-48ed-90b5-0b76f73846ab admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:42.472247 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bfe45-0bbe-48ed-90b5-0b76f73846ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:42.472563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bfe45-0bbe-48ed-90b5-0b76f73846ab admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:42.473303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bfe45-0bbe-48ed-90b5-0b76f73846ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:42.475339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78bfe45-0bbe-48ed-90b5-0b76f73846ab admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:42.510297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc1564-b2df-478e-b351-3f9c524a8331 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:42.512498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-31fc1564-b2df-478e-b351-3f9c524a8331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:42.529919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc1564-b2df-478e-b351-3f9c524a8331 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:42.536975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:42.537998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:42.538565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:42.538891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] PATH_INFO: `/v3/projects/4ff033e7c38e4be9a9c3ed8f0bca95dc` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:42.540838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff033e7c38e4be9a9c3ed8f0bca95dc)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:42.543856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:42.656900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 985/2002] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:42 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:03:42.661401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bfe45-0bbe-48ed-90b5-0b76f73846ab admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:42.678724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bfe45-0bbe-48ed-90b5-0b76f73846ab admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:42.687479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313e902-063b-4efa-be54-610225603e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:42.688595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313e902-063b-4efa-be54-610225603e66 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:42.689082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313e902-063b-4efa-be54-610225603e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:42.689593 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313e902-063b-4efa-be54-610225603e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:42.690825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313e902-063b-4efa-be54-610225603e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:42.694016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313e902-063b-4efa-be54-610225603e66 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:42.714916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 986/2003] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:03:42.716059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313e902-063b-4efa-be54-610225603e66 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:42.726490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff033e7c38e4be9a9c3ed8f0bca95dc', 'request_id': 'req-71593f68-fff1-4ac2-8443-19d6b8c40a89'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:42.734496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff033e7c38e4be9a9c3ed8f0bca95dc', 'request_id': 'req-71593f68-fff1-4ac2-8443-19d6b8c40a89'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:42.743818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313e902-063b-4efa-be54-610225603e66 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:42.845263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4fdd73-64e9-40a9-b028-4e1b55dd4b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:42.848404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4fdd73-64e9-40a9-b028-4e1b55dd4b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:42.848828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4fdd73-64e9-40a9-b028-4e1b55dd4b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:42.848828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4fdd73-64e9-40a9-b028-4e1b55dd4b6d admin admin] PATH_INFO: `/v3/projects/9373f3eee4f844f5bb16ea150d123bb7/users/d4c6f2541f944fbd9c6c5f0bf99e735f/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:42.873345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4fdd73-64e9-40a9-b028-4e1b55dd4b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9373f3eee4f844f5bb16ea150d123bb7, user_id=d4c6f2541f944fbd9c6c5f0bf99e735f, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:42.879403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4fdd73-64e9-40a9-b028-4e1b55dd4b6d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:42.895205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff033e7c38e4be9a9c3ed8f0bca95dc', 'request_id': 'req-71593f68-fff1-4ac2-8443-19d6b8c40a89'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:42.915592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] The token cache is being invalidate because project 4ff033e7c38e4be9a9c3ed8f0bca95dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:42.916059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:42.917167 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-71593f68-fff1-4ac2-8443-19d6b8c40a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:42.919203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1018/2004] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:42 2022] DELETE /identity/v3/projects/4ff033e7c38e4be9a9c3ed8f0bca95dc => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:42.924688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37cf0-f81a-4cd9-ab4b-b9e839721e8f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:42.925093 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37cf0-f81a-4cd9-ab4b-b9e839721e8f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:42.925555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37cf0-f81a-4cd9-ab4b-b9e839721e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:42.950194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e6a37cf0-f81a-4cd9-ab4b-b9e839721e8f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:42.952371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e6a37cf0-f81a-4cd9-ab4b-b9e839721e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:42.973707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1019/2005] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:42.984120 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09107f6c-9b86-46a7-b4af-e7c055e7bc9c None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:42.984676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09107f6c-9b86-46a7-b4af-e7c055e7bc9c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:42.985087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09107f6c-9b86-46a7-b4af-e7c055e7bc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.017953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-09107f6c-9b86-46a7-b4af-e7c055e7bc9c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:43.019908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-09107f6c-9b86-46a7-b4af-e7c055e7bc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:43.032709 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1020/2006] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:43.040721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09107f6c-9b86-46a7-b4af-e7c055e7bc9c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.047594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 987/2007] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:42 2022] PUT /identity/v3/projects/9373f3eee4f844f5bb16ea150d123bb7/users/d4c6f2541f944fbd9c6c5f0bf99e735f/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:43.053031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09107f6c-9b86-46a7-b4af-e7c055e7bc9c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.053347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4fdd73-64e9-40a9-b028-4e1b55dd4b6d admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.055376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7c4fdd73-64e9-40a9-b028-4e1b55dd4b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:43.139803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771adbd5-eb42-46f5-87b8-e472693c688d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.140766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771adbd5-eb42-46f5-87b8-e472693c688d admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.141215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771adbd5-eb42-46f5-87b8-e472693c688d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.141674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771adbd5-eb42-46f5-87b8-e472693c688d admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.142805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771adbd5-eb42-46f5-87b8-e472693c688d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:43.143386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771adbd5-eb42-46f5-87b8-e472693c688d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:43.146381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771adbd5-eb42-46f5-87b8-e472693c688d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:43.160145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4fdd73-64e9-40a9-b028-4e1b55dd4b6d admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.167516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af402a8-a80b-460f-adf7-7ec01101242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.168778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af402a8-a80b-460f-adf7-7ec01101242a admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.169255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af402a8-a80b-460f-adf7-7ec01101242a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.169743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af402a8-a80b-460f-adf7-7ec01101242a admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.170932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af402a8-a80b-460f-adf7-7ec01101242a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:43.173166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1021/2008] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:03:43.174193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af402a8-a80b-460f-adf7-7ec01101242a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:43.175277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771adbd5-eb42-46f5-87b8-e472693c688d admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.189636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771adbd5-eb42-46f5-87b8-e472693c688d admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.190544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 988/2009] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:43.194787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af402a8-a80b-460f-adf7-7ec01101242a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.197322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdccea3-4794-481c-951b-44506f77fe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.197839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdccea3-4794-481c-951b-44506f77fe03 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.197839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdccea3-4794-481c-951b-44506f77fe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.198661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdccea3-4794-481c-951b-44506f77fe03 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.198750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdccea3-4794-481c-951b-44506f77fe03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:43.207530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdccea3-4794-481c-951b-44506f77fe03 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:43.207870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af402a8-a80b-460f-adf7-7ec01101242a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.212787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a63b36-3a62-4788-a2c8-ee12f63930a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.213530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a63b36-3a62-4788-a2c8-ee12f63930a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.213815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a63b36-3a62-4788-a2c8-ee12f63930a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.214097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a63b36-3a62-4788-a2c8-ee12f63930a8 admin admin] PATH_INFO: `/v3/projects/9373f3eee4f844f5bb16ea150d123bb7/users/d4c6f2541f944fbd9c6c5f0bf99e735f/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.217602 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a63b36-3a62-4788-a2c8-ee12f63930a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9373f3eee4f844f5bb16ea150d123bb7, user_id=d4c6f2541f944fbd9c6c5f0bf99e735f, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:43.220454 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a63b36-3a62-4788-a2c8-ee12f63930a8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:43.239397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 989/2010] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:43 2022] PUT /identity/v3/projects/9373f3eee4f844f5bb16ea150d123bb7/users/d4c6f2541f944fbd9c6c5f0bf99e735f/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:43.242718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a63b36-3a62-4788-a2c8-ee12f63930a8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.245736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a3a63b36-3a62-4788-a2c8-ee12f63930a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:43.353962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a63b36-3a62-4788-a2c8-ee12f63930a8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.361826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.362562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.362887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.363207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] PATH_INFO: `/v3/projects/cfbbe24868084fbe85965bafadcbade7` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.364587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfbbe24868084fbe85965bafadcbade7)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:43.366675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:43.400165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1022/2011] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:43 2022] POST /identity/v3/projects => generated 375 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:43.405946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdccea3-4794-481c-951b-44506f77fe03 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.475536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdccea3-4794-481c-951b-44506f77fe03 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.480681 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cbdc1-74a0-4df9-9f5b-ff2ead447457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.481384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cbdc1-74a0-4df9-9f5b-ff2ead447457 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.481667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cbdc1-74a0-4df9-9f5b-ff2ead447457 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.481947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cbdc1-74a0-4df9-9f5b-ff2ead447457 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.482602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cbdc1-74a0-4df9-9f5b-ff2ead447457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:43.484620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cbdc1-74a0-4df9-9f5b-ff2ead447457 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:43.608986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfbbe24868084fbe85965bafadcbade7', 'request_id': 'req-72976235-62e6-4865-aeb3-0d705dcff579'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:43.618525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfbbe24868084fbe85965bafadcbade7', 'request_id': 'req-72976235-62e6-4865-aeb3-0d705dcff579'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:43.626529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1023/2012] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:43 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:03:43.629623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfbbe24868084fbe85965bafadcbade7', 'request_id': 'req-72976235-62e6-4865-aeb3-0d705dcff579'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:43.632657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cbdc1-74a0-4df9-9f5b-ff2ead447457 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.644539 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cbdc1-74a0-4df9-9f5b-ff2ead447457 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.740300 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0547a8b-caed-4f96-8ce7-e9ad0559ca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.741297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0547a8b-caed-4f96-8ce7-e9ad0559ca2d admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.741297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0547a8b-caed-4f96-8ce7-e9ad0559ca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.741554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0547a8b-caed-4f96-8ce7-e9ad0559ca2d admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.742341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0547a8b-caed-4f96-8ce7-e9ad0559ca2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:43.744156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0547a8b-caed-4f96-8ce7-e9ad0559ca2d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:43.757290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1024/2013] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:03:43.764659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0547a8b-caed-4f96-8ce7-e9ad0559ca2d admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.778051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0547a8b-caed-4f96-8ce7-e9ad0559ca2d admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.801541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] The token cache is being invalidate because project cfbbe24868084fbe85965bafadcbade7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:43.821995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:43.821995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-72976235-62e6-4865-aeb3-0d705dcff579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:43.821995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 990/2014] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:43 2022] DELETE /identity/v3/projects/cfbbe24868084fbe85965bafadcbade7 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:43.823009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdec5e9-ae80-4453-bb7e-1035afa2086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.823009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdec5e9-ae80-4453-bb7e-1035afa2086e admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.823009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdec5e9-ae80-4453-bb7e-1035afa2086e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.823009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdec5e9-ae80-4453-bb7e-1035afa2086e admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.823348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdec5e9-ae80-4453-bb7e-1035afa2086e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:43.823348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdec5e9-ae80-4453-bb7e-1035afa2086e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:43.827341 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525c3cb-db01-49d5-965f-aa2229a2de7b None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.828002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525c3cb-db01-49d5-965f-aa2229a2de7b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.828620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525c3cb-db01-49d5-965f-aa2229a2de7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.850407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1025/2015] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:43.857549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdec5e9-ae80-4453-bb7e-1035afa2086e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.858575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-c525c3cb-db01-49d5-965f-aa2229a2de7b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:43.860017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c525c3cb-db01-49d5-965f-aa2229a2de7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:43.862378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3fdec5e9-ae80-4453-bb7e-1035afa2086e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:43.880431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 991/2016] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:43.895500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdec5e9-ae80-4453-bb7e-1035afa2086e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.898508 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb120c-6290-40e4-ab42-da7f2d767944 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.899235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb120c-6290-40e4-ab42-da7f2d767944 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.899579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb120c-6290-40e4-ab42-da7f2d767944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.904111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04b59d-17f3-4a18-9266-c3f78284fe86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.905460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b59d-17f3-4a18-9266-c3f78284fe86 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.906261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b59d-17f3-4a18-9266-c3f78284fe86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.907033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b59d-17f3-4a18-9266-c3f78284fe86 admin admin] PATH_INFO: `/v3/projects/9cba13776ab74f93b1b3ff1550fff39b/users/9f060b09b83b44939aee161095301d63/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.928288 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-08cb120c-6290-40e4-ab42-da7f2d767944 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:43.929864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-08cb120c-6290-40e4-ab42-da7f2d767944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:43.934203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04b59d-17f3-4a18-9266-c3f78284fe86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cba13776ab74f93b1b3ff1550fff39b, user_id=9f060b09b83b44939aee161095301d63, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:43.936947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04b59d-17f3-4a18-9266-c3f78284fe86 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:43.939036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 992/2017] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:43.944622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb120c-6290-40e4-ab42-da7f2d767944 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.954281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb120c-6290-40e4-ab42-da7f2d767944 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.959669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c32330-6639-477f-8696-ecb09f08d1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.960324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c32330-6639-477f-8696-ecb09f08d1cb admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.960615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c32330-6639-477f-8696-ecb09f08d1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.961062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c32330-6639-477f-8696-ecb09f08d1cb admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.961727 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c32330-6639-477f-8696-ecb09f08d1cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:03:43.962030 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c32330-6639-477f-8696-ecb09f08d1cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:43.963657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c32330-6639-477f-8696-ecb09f08d1cb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:43.976956 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 993/2018] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:03:43.981617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c32330-6639-477f-8696-ecb09f08d1cb admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:43.992228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c32330-6639-477f-8696-ecb09f08d1cb admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:43.997074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad010c56-9971-4a9e-83f1-e151139f09aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:43.997687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad010c56-9971-4a9e-83f1-e151139f09aa admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:43.997965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad010c56-9971-4a9e-83f1-e151139f09aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:43.998235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad010c56-9971-4a9e-83f1-e151139f09aa admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:43.998871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad010c56-9971-4a9e-83f1-e151139f09aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:44.000857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad010c56-9971-4a9e-83f1-e151139f09aa admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:44.295931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:44.305157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 994/2019] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:43 2022] POST /identity/v3/projects => generated 383 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:44.305506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1026/2020] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:43 2022] PUT /identity/v3/projects/9cba13776ab74f93b1b3ff1550fff39b/users/9f060b09b83b44939aee161095301d63/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:44.311577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad010c56-9971-4a9e-83f1-e151139f09aa admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:44.325078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd5c97-117a-4443-9eb9-ce8be2f21ad8 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:44.325530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd5c97-117a-4443-9eb9-ce8be2f21ad8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:44.326245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd5c97-117a-4443-9eb9-ce8be2f21ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:44.375323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4cdd5c97-117a-4443-9eb9-ce8be2f21ad8 None None] MFA Rules not processed for user `9f060b09b83b44939aee161095301d63`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:44.378081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4cdd5c97-117a-4443-9eb9-ce8be2f21ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:44.404756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad010c56-9971-4a9e-83f1-e151139f09aa admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:44.416088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd2c54-96c0-408c-aa0c-42e95cb4eae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:44.418073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd2c54-96c0-408c-aa0c-42e95cb4eae2 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:44.418624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd2c54-96c0-408c-aa0c-42e95cb4eae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:44.419139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd2c54-96c0-408c-aa0c-42e95cb4eae2 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:44.420408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd2c54-96c0-408c-aa0c-42e95cb4eae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:44.425302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd2c54-96c0-408c-aa0c-42e95cb4eae2 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:44.458190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:44.513200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1027/2021] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:44.527466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd5c97-117a-4443-9eb9-ce8be2f21ad8 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:44.529924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4cdd5c97-117a-4443-9eb9-ce8be2f21ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:44.609038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 995/2022] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:44 2022] POST /identity/v3/users => generated 292 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:44.614914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd2c54-96c0-408c-aa0c-42e95cb4eae2 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:44.627152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd2c54-96c0-408c-aa0c-42e95cb4eae2 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:44.633819 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86e22c-e279-4781-9318-92bc8535d0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:44.634746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86e22c-e279-4781-9318-92bc8535d0ba admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:44.635162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86e22c-e279-4781-9318-92bc8535d0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:44.635566 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86e22c-e279-4781-9318-92bc8535d0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:44.636418 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86e22c-e279-4781-9318-92bc8535d0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:44.638172 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86e22c-e279-4781-9318-92bc8535d0ba admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:44.657209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 996/2023] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:44.659577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86e22c-e279-4781-9318-92bc8535d0ba admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:44.674425 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd5c97-117a-4443-9eb9-ce8be2f21ad8 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:44.678249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86e22c-e279-4781-9318-92bc8535d0ba admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:44.679955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e8341-e14e-4024-baf4-0af9d9ceebf7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454fb811f6e34e1d881d4948ede058b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:44.681048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e8341-e14e-4024-baf4-0af9d9ceebf7 service glance] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:44.681688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e8341-e14e-4024-baf4-0af9d9ceebf7 service glance] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:44.682116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e8341-e14e-4024-baf4-0af9d9ceebf7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:44.686803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65ef1f-7a56-4932-8d51-4265e8ca00a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:44.687638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ef1f-7a56-4932-8d51-4265e8ca00a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:44.691308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ef1f-7a56-4932-8d51-4265e8ca00a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:44.691308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ef1f-7a56-4932-8d51-4265e8ca00a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:44.691308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65ef1f-7a56-4932-8d51-4265e8ca00a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:44.691308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65ef1f-7a56-4932-8d51-4265e8ca00a3 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:44.700749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e8341-e14e-4024-baf4-0af9d9ceebf7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:44.704643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e8341-e14e-4024-baf4-0af9d9ceebf7 service glance] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:44.707007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 997/2024] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:44.712100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65ef1f-7a56-4932-8d51-4265e8ca00a3 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:44.726475 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65ef1f-7a56-4932-8d51-4265e8ca00a3 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:44.731185 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1028/2025] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:03:44.732300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4313125-dfb7-4db6-b081-c4d2fc798922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:44.733256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4313125-dfb7-4db6-b081-c4d2fc798922 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:44.733723 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4313125-dfb7-4db6-b081-c4d2fc798922 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:44.734101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4313125-dfb7-4db6-b081-c4d2fc798922 admin admin] PATH_INFO: `/v3/projects/61e730edaa844158b1d1a1470fc27d10/users/c9a857096e2e4a659697559ea410bcbe/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:44.755389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4313125-dfb7-4db6-b081-c4d2fc798922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e730edaa844158b1d1a1470fc27d10, user_id=c9a857096e2e4a659697559ea410bcbe, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:44.761455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4313125-dfb7-4db6-b081-c4d2fc798922 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:44.902919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 998/2026] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:44 2022] PUT /identity/v3/projects/61e730edaa844158b1d1a1470fc27d10/users/c9a857096e2e4a659697559ea410bcbe/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:44.919932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d12203-c872-49ff-9c72-ca7020db071b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:44.920562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d12203-c872-49ff-9c72-ca7020db071b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:44.920890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d12203-c872-49ff-9c72-ca7020db071b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:44.971172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-52d12203-c872-49ff-9c72-ca7020db071b None None] MFA Rules not processed for user `c9a857096e2e4a659697559ea410bcbe`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:44.973343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-52d12203-c872-49ff-9c72-ca7020db071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:45.118304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1029/2027] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:45.123126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4313125-dfb7-4db6-b081-c4d2fc798922 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:45.211137 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4313125-dfb7-4db6-b081-c4d2fc798922 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:45.213527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2b6a4-f1c9-4d32-b5ef-f07f065c86e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:45.214185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2b6a4-f1c9-4d32-b5ef-f07f065c86e2 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:45.214523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2b6a4-f1c9-4d32-b5ef-f07f065c86e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:45.214802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2b6a4-f1c9-4d32-b5ef-f07f065c86e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:45.216435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2b6a4-f1c9-4d32-b5ef-f07f065c86e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:45.217738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2b6a4-f1c9-4d32-b5ef-f07f065c86e2 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:45.420280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 999/2028] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:45 2022] POST /identity/v3/projects => generated 385 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:45.425633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d12203-c872-49ff-9c72-ca7020db071b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:45.438865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d12203-c872-49ff-9c72-ca7020db071b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:45.548669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2bcbb-4929-4044-ba08-1c7903f9323c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:45.549899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2bcbb-4929-4044-ba08-1c7903f9323c admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:45.555017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2bcbb-4929-4044-ba08-1c7903f9323c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:45.555017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2bcbb-4929-4044-ba08-1c7903f9323c admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:45.555017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2bcbb-4929-4044-ba08-1c7903f9323c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:45.555876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2bcbb-4929-4044-ba08-1c7903f9323c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:45.605954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7858b6b-b128-420c-8943-e8cc1859de76 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:45.606379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7858b6b-b128-420c-8943-e8cc1859de76 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:45.606801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7858b6b-b128-420c-8943-e8cc1859de76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:45.656760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e7858b6b-b128-420c-8943-e8cc1859de76 None None] MFA Rules not processed for user `d4c6f2541f944fbd9c6c5f0bf99e735f`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:45.659591 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e7858b6b-b128-420c-8943-e8cc1859de76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:45.717870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1030/2029] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:45 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:03:45.725554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2bcbb-4929-4044-ba08-1c7903f9323c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:45.745607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2bcbb-4929-4044-ba08-1c7903f9323c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:45.754776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406e727-4e3f-4026-986e-ded16944e502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:45.756132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406e727-4e3f-4026-986e-ded16944e502 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:45.756851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406e727-4e3f-4026-986e-ded16944e502 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:45.757543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406e727-4e3f-4026-986e-ded16944e502 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:45.758978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406e727-4e3f-4026-986e-ded16944e502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:45.764330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406e727-4e3f-4026-986e-ded16944e502 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:45.786297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1031/2030] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:45.806089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406e727-4e3f-4026-986e-ded16944e502 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:45.838612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406e727-4e3f-4026-986e-ded16944e502 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:45.847725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c656ae5-f486-40ec-99f8-34994f566376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:45.848772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c656ae5-f486-40ec-99f8-34994f566376 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:45.849255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c656ae5-f486-40ec-99f8-34994f566376 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:45.849713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c656ae5-f486-40ec-99f8-34994f566376 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:45.850881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c656ae5-f486-40ec-99f8-34994f566376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:45.853747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c656ae5-f486-40ec-99f8-34994f566376 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:45.880812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1032/2031] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:45.882968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c656ae5-f486-40ec-99f8-34994f566376 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:45.892224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1000/2032] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:45.907116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c656ae5-f486-40ec-99f8-34994f566376 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:45.915457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df28309-5396-41a0-bd34-cedbcc7753c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:45.916663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df28309-5396-41a0-bd34-cedbcc7753c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:45.917133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df28309-5396-41a0-bd34-cedbcc7753c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:45.917598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df28309-5396-41a0-bd34-cedbcc7753c5 admin admin] PATH_INFO: `/v3/projects/8250850dcdef46708643dac707e210d7/users/b1f66c0f272a4157bab384d30786dd73/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:45.950839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7858b6b-b128-420c-8943-e8cc1859de76 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:45.952711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e7858b6b-b128-420c-8943-e8cc1859de76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:45.959568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df28309-5396-41a0-bd34-cedbcc7753c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8250850dcdef46708643dac707e210d7, user_id=b1f66c0f272a4157bab384d30786dd73, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:45.961872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:45.965238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df28309-5396-41a0-bd34-cedbcc7753c5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:46.030494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7858b6b-b128-420c-8943-e8cc1859de76 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:46.034595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5cc91c-b6ee-41b6-a230-f12d372c765f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:46.035289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cc91c-b6ee-41b6-a230-f12d372c765f service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:46.035571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cc91c-b6ee-41b6-a230-f12d372c765f service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:46.035849 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cc91c-b6ee-41b6-a230-f12d372c765f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:46.037785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cb5cc91c-b6ee-41b6-a230-f12d372c765f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:46.119175 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cc91c-b6ee-41b6-a230-f12d372c765f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:46.120889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cc91c-b6ee-41b6-a230-f12d372c765f service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:46.128045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1033/2033] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:45 2022] PUT /identity/v3/projects/8250850dcdef46708643dac707e210d7/users/b1f66c0f272a4157bab384d30786dd73/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:46.131370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dafc0-7865-4ac9-b5de-67e8f84b7489 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:46.131712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dafc0-7865-4ac9-b5de-67e8f84b7489 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:46.132013 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dafc0-7865-4ac9-b5de-67e8f84b7489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:46.132583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1001/2034] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:03:46.167156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-da9dafc0-7865-4ac9-b5de-67e8f84b7489 None None] MFA Rules not processed for user `b1f66c0f272a4157bab384d30786dd73`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:46.168520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-da9dafc0-7865-4ac9-b5de-67e8f84b7489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:46.280030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1034/2035] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:46.286715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5cc91c-b6ee-41b6-a230-f12d372c765f service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:46.301254 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5cc91c-b6ee-41b6-a230-f12d372c765f service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:46.407421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564487-1b5a-489d-acd0-f57c709d9077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:46.408149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564487-1b5a-489d-acd0-f57c709d9077 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:46.408664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564487-1b5a-489d-acd0-f57c709d9077 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:46.409120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564487-1b5a-489d-acd0-f57c709d9077 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:46.410430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564487-1b5a-489d-acd0-f57c709d9077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:46.414046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564487-1b5a-489d-acd0-f57c709d9077 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:46.423127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dafc0-7865-4ac9-b5de-67e8f84b7489 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:46.426350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-da9dafc0-7865-4ac9-b5de-67e8f84b7489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:46.499165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9dafc0-7865-4ac9-b5de-67e8f84b7489 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:46.504535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43abac9-b650-4f88-8f7a-d20afa61b237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:46.505323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43abac9-b650-4f88-8f7a-d20afa61b237 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:46.505642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43abac9-b650-4f88-8f7a-d20afa61b237 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:46.505958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43abac9-b650-4f88-8f7a-d20afa61b237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:46.507928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e43abac9-b650-4f88-8f7a-d20afa61b237 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:46.561798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1002/2036] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:03:46 2022] POST /identity/v3/projects => generated 385 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:46.563191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564487-1b5a-489d-acd0-f57c709d9077 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:46.598936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43abac9-b650-4f88-8f7a-d20afa61b237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:46.601820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43abac9-b650-4f88-8f7a-d20afa61b237 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:46.618035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1035/2037] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:46.659666 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564487-1b5a-489d-acd0-f57c709d9077 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:46.670483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad776fe9-e11f-4dd1-8937-6af801470ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:46.671463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad776fe9-e11f-4dd1-8937-6af801470ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:46.671940 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad776fe9-e11f-4dd1-8937-6af801470ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:46.672456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad776fe9-e11f-4dd1-8937-6af801470ff3 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:46.673589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad776fe9-e11f-4dd1-8937-6af801470ff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:46.678298 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad776fe9-e11f-4dd1-8937-6af801470ff3 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:46.836893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1003/2038] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:03:46 2022] POST /identity/v3/users => generated 299 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:03:46.841284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43abac9-b650-4f88-8f7a-d20afa61b237 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:46.865714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43abac9-b650-4f88-8f7a-d20afa61b237 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:46.873439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b61bb5-bfb3-4009-9d2d-4193c86a7a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:46.874452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b61bb5-bfb3-4009-9d2d-4193c86a7a9d admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:46.874916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b61bb5-bfb3-4009-9d2d-4193c86a7a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:46.875412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b61bb5-bfb3-4009-9d2d-4193c86a7a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:46.876626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b61bb5-bfb3-4009-9d2d-4193c86a7a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:46.879913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b61bb5-bfb3-4009-9d2d-4193c86a7a9d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:46.899871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1036/2039] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:46.904955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad776fe9-e11f-4dd1-8937-6af801470ff3 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:46.921331 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad776fe9-e11f-4dd1-8937-6af801470ff3 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:46.928365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa774c5a-f9f3-4322-b53b-3dc74019063c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:46.929793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa774c5a-f9f3-4322-b53b-3dc74019063c admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:46.930824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa774c5a-f9f3-4322-b53b-3dc74019063c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:46.931306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa774c5a-f9f3-4322-b53b-3dc74019063c admin admin] PATH_INFO: `/v3/projects/1728825365384205b0487c3c46ced621/users/b1efbce16c9d45cb93580dfa0798e32f/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:46.935066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:46.951191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa774c5a-f9f3-4322-b53b-3dc74019063c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1728825365384205b0487c3c46ced621, user_id=b1efbce16c9d45cb93580dfa0798e32f, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:46.954205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa774c5a-f9f3-4322-b53b-3dc74019063c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:47.499339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1004/2040] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:46 2022] PUT /identity/v3/projects/1728825365384205b0487c3c46ced621/users/b1efbce16c9d45cb93580dfa0798e32f/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:47.499518 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b61bb5-bfb3-4009-9d2d-4193c86a7a9d admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:47.504973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b61bb5-bfb3-4009-9d2d-4193c86a7a9d admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:47.578848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856cd17-2def-446e-83aa-fd793d190cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:47.579483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856cd17-2def-446e-83aa-fd793d190cff admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:47.579760 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856cd17-2def-446e-83aa-fd793d190cff admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:47.580035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856cd17-2def-446e-83aa-fd793d190cff admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:47.580791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856cd17-2def-446e-83aa-fd793d190cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:47.582443 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856cd17-2def-446e-83aa-fd793d190cff admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:47.594129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1037/2041] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:03:47.599084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa774c5a-f9f3-4322-b53b-3dc74019063c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:47.611444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa774c5a-f9f3-4322-b53b-3dc74019063c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:47.617668 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b96d51-efd8-4c6c-8669-174cd0e794cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:47.618443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96d51-efd8-4c6c-8669-174cd0e794cd admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:47.618718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96d51-efd8-4c6c-8669-174cd0e794cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:47.619055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96d51-efd8-4c6c-8669-174cd0e794cd admin admin] PATH_INFO: `/v3/projects/1728825365384205b0487c3c46ced621/users/b1efbce16c9d45cb93580dfa0798e32f/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:47.621313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b96d51-efd8-4c6c-8669-174cd0e794cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1728825365384205b0487c3c46ced621, user_id=b1efbce16c9d45cb93580dfa0798e32f, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:47.624161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b96d51-efd8-4c6c-8669-174cd0e794cd admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:47.641882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1005/2042] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:03:47 2022] PUT /identity/v3/projects/1728825365384205b0487c3c46ced621/users/b1efbce16c9d45cb93580dfa0798e32f/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:47.660221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffecad-311b-43de-8f2f-25ba72fa7ec4 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:47.660537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffecad-311b-43de-8f2f-25ba72fa7ec4 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:47.660763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffecad-311b-43de-8f2f-25ba72fa7ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:47.694254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-90ffecad-311b-43de-8f2f-25ba72fa7ec4 None None] MFA Rules not processed for user `b1efbce16c9d45cb93580dfa0798e32f`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:47.700302 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-90ffecad-311b-43de-8f2f-25ba72fa7ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:47.817131 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1038/2043] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:03:48.086441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b96d51-efd8-4c6c-8669-174cd0e794cd admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:48.192898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b96d51-efd8-4c6c-8669-174cd0e794cd admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:48.192898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf82a5-dfab-465b-a8fb-c53abc7914d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:48.192898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf82a5-dfab-465b-a8fb-c53abc7914d2 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:48.192898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf82a5-dfab-465b-a8fb-c53abc7914d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:48.193840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf82a5-dfab-465b-a8fb-c53abc7914d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:48.293475 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf82a5-dfab-465b-a8fb-c53abc7914d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:48.295019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf82a5-dfab-465b-a8fb-c53abc7914d2 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:48.307514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1006/2044] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:03:48.821037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:48.998651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:49.778035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffecad-311b-43de-8f2f-25ba72fa7ec4 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:49.800937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffecad-311b-43de-8f2f-25ba72fa7ec4 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:49.809792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00f86e-46a9-46a6-bfb9-c4f9796eba1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:49.811105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f86e-46a9-46a6-bfb9-c4f9796eba1c service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:49.811446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f86e-46a9-46a6-bfb9-c4f9796eba1c service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:49.812332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f86e-46a9-46a6-bfb9-c4f9796eba1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:49.949354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00f86e-46a9-46a6-bfb9-c4f9796eba1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:49.952105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00f86e-46a9-46a6-bfb9-c4f9796eba1c service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:49.973727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1039/2045] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:50.008669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:50.301583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:51.354317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf82a5-dfab-465b-a8fb-c53abc7914d2 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:51.365928 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf82a5-dfab-465b-a8fb-c53abc7914d2 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:51.370062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39553cf-4d9d-4be7-a927-b96e21f5bdc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:51.370701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39553cf-4d9d-4be7-a927-b96e21f5bdc4 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:51.370985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39553cf-4d9d-4be7-a927-b96e21f5bdc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:51.371264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39553cf-4d9d-4be7-a927-b96e21f5bdc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:51.457982 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39553cf-4d9d-4be7-a927-b96e21f5bdc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:51.460110 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39553cf-4d9d-4be7-a927-b96e21f5bdc4 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:51.471568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1007/2046] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:51.563995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:51.736422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:52.763005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:53.834418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:54.261170 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:54.822260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00f86e-46a9-46a6-bfb9-c4f9796eba1c service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:54.824696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3a00f86e-46a9-46a6-bfb9-c4f9796eba1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:54.959147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39553cf-4d9d-4be7-a927-b96e21f5bdc4 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:54.975621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39553cf-4d9d-4be7-a927-b96e21f5bdc4 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:54.981688 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c3e5f-1a5f-4de5-b6a3-edc400762c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:54.982747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c3e5f-1a5f-4de5-b6a3-edc400762c3f service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:54.983225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c3e5f-1a5f-4de5-b6a3-edc400762c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:54.983721 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c3e5f-1a5f-4de5-b6a3-edc400762c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:54.991324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00f86e-46a9-46a6-bfb9-c4f9796eba1c service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:54.998110 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:54.999984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:54.999984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:54.999984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] PATH_INFO: `/v3/users/359219acf33d4801ba241c6c222fb902` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:55.001404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=359219acf33d4801ba241c6c222fb902)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:55.002831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c3e5f-1a5f-4de5-b6a3-edc400762c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:55.003900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:55.005674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c3e5f-1a5f-4de5-b6a3-edc400762c3f service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:55.022944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1008/2047] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:55.427993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '359219acf33d4801ba241c6c222fb902'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:55.428796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '359219acf33d4801ba241c6c222fb902'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:55.439055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '359219acf33d4801ba241c6c222fb902'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:55.439860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:55.441017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '359219acf33d4801ba241c6c222fb902'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:55.545488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '359219acf33d4801ba241c6c222fb902'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:55.560103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:55.577659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1040/2048] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:54 2022] DELETE /identity/v3/users/359219acf33d4801ba241c6c222fb902 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:55.584518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833a9cd-4d69-4650-9d69-aa40a9f9bcfa None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:55.585120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833a9cd-4d69-4650-9d69-aa40a9f9bcfa None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:55.586185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833a9cd-4d69-4650-9d69-aa40a9f9bcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:55.620754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7833a9cd-4d69-4650-9d69-aa40a9f9bcfa None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:03:55.622266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7833a9cd-4d69-4650-9d69-aa40a9f9bcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:55.709242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1009/2049] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:03:55.720740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:55.722343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:55.809379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61c8c5-3cd9-4431-bb63-2a1e0b27b1c1 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:55.816757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cff91c-d8ae-4202-9cd5-038c9cace9f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:55.817839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cff91c-d8ae-4202-9cd5-038c9cace9f3 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:55.818350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cff91c-d8ae-4202-9cd5-038c9cace9f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:55.818851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cff91c-d8ae-4202-9cd5-038c9cace9f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:55.838438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cff91c-d8ae-4202-9cd5-038c9cace9f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:55.840093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:55.841330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cff91c-d8ae-4202-9cd5-038c9cace9f3 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:55.854520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1041/2050] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:03:56.036081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:56.084137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:56.685409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833a9cd-4d69-4650-9d69-aa40a9f9bcfa None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:56.689572 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7833a9cd-4d69-4650-9d69-aa40a9f9bcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:56.709488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833a9cd-4d69-4650-9d69-aa40a9f9bcfa None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:56.714326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:56.716868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:56.718096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:56.721470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:56.721470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] PATH_INFO: `/v3/projects/9ede7129ace74745b56f13bf58a590ed` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:56.721470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ede7129ace74745b56f13bf58a590ed)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:56.723623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:56.884410 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ede7129ace74745b56f13bf58a590ed', 'request_id': 'req-29e284f2-f7a9-4e57-8d48-ea4090271bb0'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:56.895489 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ede7129ace74745b56f13bf58a590ed', 'request_id': 'req-29e284f2-f7a9-4e57-8d48-ea4090271bb0'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:56.908811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ede7129ace74745b56f13bf58a590ed', 'request_id': 'req-29e284f2-f7a9-4e57-8d48-ea4090271bb0'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:57.004811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] The token cache is being invalidate because project 9ede7129ace74745b56f13bf58a590ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:57.005266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:57.006049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:57.008347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1010/2051] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:56 2022] DELETE /identity/v3/projects/9ede7129ace74745b56f13bf58a590ed => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:03:57.012508 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cff91c-d8ae-4202-9cd5-038c9cace9f3 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:57.023516 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cff91c-d8ae-4202-9cd5-038c9cace9f3 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:57.026114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:57.091822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:57.095773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:57.095773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:57.095773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] PATH_INFO: `/v3/users/a615dae8e5d946c88e5f13238a3c192e` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:57.095773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a615dae8e5d946c88e5f13238a3c192e)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:57.096551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:57.378912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a615dae8e5d946c88e5f13238a3c192e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:57.379768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a615dae8e5d946c88e5f13238a3c192e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:57.388471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a615dae8e5d946c88e5f13238a3c192e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:57.389242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a615dae8e5d946c88e5f13238a3c192e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:57.511167 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a615dae8e5d946c88e5f13238a3c192e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:57.522830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:57.536641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1042/2052] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:57 2022] DELETE /identity/v3/users/a615dae8e5d946c88e5f13238a3c192e => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:57.971635 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:58.149070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:58.151857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:58.174698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e284f2-f7a9-4e57-8d48-ea4090271bb0 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:58.221351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:58.252779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:58.253437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:58.253724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:58.254000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] PATH_INFO: `/v3/projects/1d28adfb08a741638ea9702916b24194` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:58.255242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d28adfb08a741638ea9702916b24194)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:58.257408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:58.408407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d28adfb08a741638ea9702916b24194', 'request_id': 'req-e75e0b88-2ac6-429f-ae67-2172833840b3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:58.417848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d28adfb08a741638ea9702916b24194', 'request_id': 'req-e75e0b88-2ac6-429f-ae67-2172833840b3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:58.426734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d28adfb08a741638ea9702916b24194', 'request_id': 'req-e75e0b88-2ac6-429f-ae67-2172833840b3'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:58.443951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:58.534265 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] The token cache is being invalidate because project 1d28adfb08a741638ea9702916b24194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:03:58.534566 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:58.535244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:58.536841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1011/2053] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:03:58 2022] DELETE /identity/v3/projects/1d28adfb08a741638ea9702916b24194 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:58.541744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:03:58.544298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:03:58.569079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24935e-e0de-4a52-a408-cecd0397a265 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:03:58.646235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:03:58.647054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:03:58.647337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:03:58.647619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] PATH_INFO: `/v3/users/6c5c0983061d423690d951643b92a5b5` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:03:58.649190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c5c0983061d423690d951643b92a5b5)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:03:58.653087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:03:58.899463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:59.107653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c5c0983061d423690d951643b92a5b5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:59.108874 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c5c0983061d423690d951643b92a5b5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:59.125541 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c5c0983061d423690d951643b92a5b5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:59.126718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c5c0983061d423690d951643b92a5b5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:59.268564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:59.334958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e82b6560-6e6e-474b-b91a-201aa3464e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c5c0983061d423690d951643b92a5b5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:03:59.370399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:59.373296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1043/2054] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:03:58 2022] DELETE /identity/v3/users/6c5c0983061d423690d951643b92a5b5 => generated 0 bytes in 832 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:03:59.764571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:03:59.860114 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:00.125430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:00.128379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:00.143438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e0b88-2ac6-429f-ae67-2172833840b3 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:00.218034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:00.218963 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:00.219240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:00.219514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] PATH_INFO: `/v3/projects/b2425c01d8014202b624ce72c24b8baa` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:00.220853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2425c01d8014202b624ce72c24b8baa)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:00.222936 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:00.516698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2425c01d8014202b624ce72c24b8baa', 'request_id': 'req-d403e309-96e9-4a04-82bc-8bada2058bfd'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:00.526425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2425c01d8014202b624ce72c24b8baa', 'request_id': 'req-d403e309-96e9-4a04-82bc-8bada2058bfd'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:00.536131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2425c01d8014202b624ce72c24b8baa', 'request_id': 'req-d403e309-96e9-4a04-82bc-8bada2058bfd'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:00.691838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] The token cache is being invalidate because project b2425c01d8014202b624ce72c24b8baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:00.691838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:00.691838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d403e309-96e9-4a04-82bc-8bada2058bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:00.691838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1012/2055] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:00 2022] DELETE /identity/v3/projects/b2425c01d8014202b624ce72c24b8baa => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:00.693022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a9dac-adf4-45cc-8279-2bdde227f17e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:00.693746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a9dac-adf4-45cc-8279-2bdde227f17e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:00.693746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a9dac-adf4-45cc-8279-2bdde227f17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:00.716763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-fe5a9dac-adf4-45cc-8279-2bdde227f17e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:00.718148 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fe5a9dac-adf4-45cc-8279-2bdde227f17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:00.760338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:00.797727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1044/2056] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:00.809435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c88dd3-d119-4c65-8ff0-a4da458b8c30 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:00.809893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c88dd3-d119-4c65-8ff0-a4da458b8c30 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:00.810380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c88dd3-d119-4c65-8ff0-a4da458b8c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:00.834127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a7c88dd3-d119-4c65-8ff0-a4da458b8c30 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:00.835413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a7c88dd3-d119-4c65-8ff0-a4da458b8c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:00.844582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1013/2057] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:04:00.852597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a9dac-adf4-45cc-8279-2bdde227f17e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:00.868301 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a9dac-adf4-45cc-8279-2bdde227f17e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:00.874270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:00.885257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5aab3-de1e-4482-9ab0-6317629cc387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:00.885836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5aab3-de1e-4482-9ab0-6317629cc387 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:00.886290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5aab3-de1e-4482-9ab0-6317629cc387 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:00.886710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5aab3-de1e-4482-9ab0-6317629cc387 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:00.887763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5aab3-de1e-4482-9ab0-6317629cc387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:00.888301 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5aab3-de1e-4482-9ab0-6317629cc387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:00.891240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5aab3-de1e-4482-9ab0-6317629cc387 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:00.912895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1045/2058] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:00.921870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c88dd3-d119-4c65-8ff0-a4da458b8c30 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:00.931902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c88dd3-d119-4c65-8ff0-a4da458b8c30 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:00.937542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0594f0-c3da-4cbc-a837-c7c6b1ae61bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:00.938213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0594f0-c3da-4cbc-a837-c7c6b1ae61bb admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:00.938486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0594f0-c3da-4cbc-a837-c7c6b1ae61bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:00.938988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0594f0-c3da-4cbc-a837-c7c6b1ae61bb admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:00.939864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0594f0-c3da-4cbc-a837-c7c6b1ae61bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:00.950381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0594f0-c3da-4cbc-a837-c7c6b1ae61bb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:01.070440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5aab3-de1e-4482-9ab0-6317629cc387 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:01.080850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1014/2059] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:00 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:01.097484 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5aab3-de1e-4482-9ab0-6317629cc387 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:01.221989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06223e92-3385-4600-a57b-0c36f7825e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:01.222783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06223e92-3385-4600-a57b-0c36f7825e86 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:01.223249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06223e92-3385-4600-a57b-0c36f7825e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:01.223626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06223e92-3385-4600-a57b-0c36f7825e86 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:01.224476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06223e92-3385-4600-a57b-0c36f7825e86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:01.226613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06223e92-3385-4600-a57b-0c36f7825e86 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:01.387620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:01.398884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1046/2060] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:01 2022] POST /identity/v3/users => generated 292 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:01.403170 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0594f0-c3da-4cbc-a837-c7c6b1ae61bb admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:01.415643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0594f0-c3da-4cbc-a837-c7c6b1ae61bb admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:01.421522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e09de-96c6-49f4-a7bc-c949ad89dafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:01.422262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e09de-96c6-49f4-a7bc-c949ad89dafe admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:01.422629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e09de-96c6-49f4-a7bc-c949ad89dafe admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:01.423063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e09de-96c6-49f4-a7bc-c949ad89dafe admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:01.424556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e09de-96c6-49f4-a7bc-c949ad89dafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:01.426671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e09de-96c6-49f4-a7bc-c949ad89dafe admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:01.439748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1015/2061] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:01.446648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06223e92-3385-4600-a57b-0c36f7825e86 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:01.461100 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06223e92-3385-4600-a57b-0c36f7825e86 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:01.466565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0e284-f6ef-4ba8-9eb5-3c90b2955e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:01.467204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0e284-f6ef-4ba8-9eb5-3c90b2955e5e admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:01.467489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0e284-f6ef-4ba8-9eb5-3c90b2955e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:01.467772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0e284-f6ef-4ba8-9eb5-3c90b2955e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:01.468502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0e284-f6ef-4ba8-9eb5-3c90b2955e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:01.470212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0e284-f6ef-4ba8-9eb5-3c90b2955e5e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:01.482893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1047/2062] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:01.489972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e09de-96c6-49f4-a7bc-c949ad89dafe admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:01.502675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e09de-96c6-49f4-a7bc-c949ad89dafe admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:01.508288 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e5dfd-7dc5-46cf-bc30-1ef5d4c552a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:01.509493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e5dfd-7dc5-46cf-bc30-1ef5d4c552a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:01.510016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e5dfd-7dc5-46cf-bc30-1ef5d4c552a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:01.510490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e5dfd-7dc5-46cf-bc30-1ef5d4c552a5 admin admin] PATH_INFO: `/v3/projects/e83502415eb6488c9ea4315ef49a4e9a/users/8e9f17e2252c4d04b672454aa550dafe/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:01.530070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e5dfd-7dc5-46cf-bc30-1ef5d4c552a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83502415eb6488c9ea4315ef49a4e9a, user_id=8e9f17e2252c4d04b672454aa550dafe, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:01.533893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e5dfd-7dc5-46cf-bc30-1ef5d4c552a5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:01.621840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1016/2063] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:01 2022] PUT /identity/v3/projects/e83502415eb6488c9ea4315ef49a4e9a/users/8e9f17e2252c4d04b672454aa550dafe/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:01.635041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414772e1-12a8-4a8d-bfa5-39d2fd21d645 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:01.635517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414772e1-12a8-4a8d-bfa5-39d2fd21d645 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:01.635867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414772e1-12a8-4a8d-bfa5-39d2fd21d645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:01.672029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-414772e1-12a8-4a8d-bfa5-39d2fd21d645 None None] MFA Rules not processed for user `8e9f17e2252c4d04b672454aa550dafe`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:01.673503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-414772e1-12a8-4a8d-bfa5-39d2fd21d645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:01.780726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1048/2064] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:01.787451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e5dfd-7dc5-46cf-bc30-1ef5d4c552a5 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:01.870576 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:01.908429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e5dfd-7dc5-46cf-bc30-1ef5d4c552a5 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:01.914497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b4dbf-b522-4822-ae81-70bed67757a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:01.914497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b4dbf-b522-4822-ae81-70bed67757a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:01.914822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b4dbf-b522-4822-ae81-70bed67757a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:01.915014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b4dbf-b522-4822-ae81-70bed67757a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:01.915684 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b4dbf-b522-4822-ae81-70bed67757a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:01.917822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b4dbf-b522-4822-ae81-70bed67757a1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:02.081152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1017/2065] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:01 2022] POST /identity/v3/projects => generated 383 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:02.089315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414772e1-12a8-4a8d-bfa5-39d2fd21d645 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:02.103273 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414772e1-12a8-4a8d-bfa5-39d2fd21d645 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:02.227119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dd055-e98b-4957-b623-c46e71d0d3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:02.228348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dd055-e98b-4957-b623-c46e71d0d3db admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:02.228840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dd055-e98b-4957-b623-c46e71d0d3db admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:02.229375 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dd055-e98b-4957-b623-c46e71d0d3db admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:02.230608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5dd055-e98b-4957-b623-c46e71d0d3db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:02.234233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5dd055-e98b-4957-b623-c46e71d0d3db admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:02.459189 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1049/2066] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:02 2022] POST /identity/v3/users => generated 298 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:04:02.461799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b4dbf-b522-4822-ae81-70bed67757a1 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:02.473921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:02.484430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b4dbf-b522-4822-ae81-70bed67757a1 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:02.501281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9d991-b3d3-4847-b312-5ee7a4428212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:02.501281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9d991-b3d3-4847-b312-5ee7a4428212 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:02.501281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9d991-b3d3-4847-b312-5ee7a4428212 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:02.501281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9d991-b3d3-4847-b312-5ee7a4428212 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:02.501806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9d991-b3d3-4847-b312-5ee7a4428212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:02.501806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9d991-b3d3-4847-b312-5ee7a4428212 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:02.531517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dd055-e98b-4957-b623-c46e71d0d3db admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:02.541650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1018/2067] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:02.546346 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:02.552575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dd055-e98b-4957-b623-c46e71d0d3db admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:02.562627 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac064e1-5eee-4bc8-8e1d-2ad0ff0a5075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:02.564838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac064e1-5eee-4bc8-8e1d-2ad0ff0a5075 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:02.565478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac064e1-5eee-4bc8-8e1d-2ad0ff0a5075 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:02.567004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac064e1-5eee-4bc8-8e1d-2ad0ff0a5075 admin admin] PATH_INFO: `/v3/projects/de6db739a16648b2bfd99977f99f96e3/users/f05ebe8158f441cab7452c3f4d14d1d6/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:02.593128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac064e1-5eee-4bc8-8e1d-2ad0ff0a5075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de6db739a16648b2bfd99977f99f96e3, user_id=f05ebe8158f441cab7452c3f4d14d1d6, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:02.596057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac064e1-5eee-4bc8-8e1d-2ad0ff0a5075 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:02.695690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1050/2068] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:02 2022] PUT /identity/v3/projects/de6db739a16648b2bfd99977f99f96e3/users/f05ebe8158f441cab7452c3f4d14d1d6/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:02.701162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9d991-b3d3-4847-b312-5ee7a4428212 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:02.714606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9d991-b3d3-4847-b312-5ee7a4428212 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:02.899461 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579ef89-ca30-47ef-8d6d-9ead74ea184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:02.902093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579ef89-ca30-47ef-8d6d-9ead74ea184c admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:02.903460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579ef89-ca30-47ef-8d6d-9ead74ea184c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:02.904486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579ef89-ca30-47ef-8d6d-9ead74ea184c admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:02.905984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579ef89-ca30-47ef-8d6d-9ead74ea184c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:02.911685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579ef89-ca30-47ef-8d6d-9ead74ea184c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:02.944485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1019/2069] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:04:02.948231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac064e1-5eee-4bc8-8e1d-2ad0ff0a5075 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:02.969733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac064e1-5eee-4bc8-8e1d-2ad0ff0a5075 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:02.981401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b30f6-e76c-4c84-b9f0-505f2265aa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:02.983703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b30f6-e76c-4c84-b9f0-505f2265aa98 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:02.984307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b30f6-e76c-4c84-b9f0-505f2265aa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:02.984874 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b30f6-e76c-4c84-b9f0-505f2265aa98 admin admin] PATH_INFO: `/v3/projects/de6db739a16648b2bfd99977f99f96e3/users/f05ebe8158f441cab7452c3f4d14d1d6/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:02.990871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b30f6-e76c-4c84-b9f0-505f2265aa98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de6db739a16648b2bfd99977f99f96e3, user_id=f05ebe8158f441cab7452c3f4d14d1d6, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:03.002982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b30f6-e76c-4c84-b9f0-505f2265aa98 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:03.023127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1051/2070] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:02 2022] PUT /identity/v3/projects/de6db739a16648b2bfd99977f99f96e3/users/f05ebe8158f441cab7452c3f4d14d1d6/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:03.040811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cb488-993a-40b6-ada4-4a94b80f33e2 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:03.041446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cb488-993a-40b6-ada4-4a94b80f33e2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:03.042261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cb488-993a-40b6-ada4-4a94b80f33e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:03.098860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-674cb488-993a-40b6-ada4-4a94b80f33e2 None None] MFA Rules not processed for user `f05ebe8158f441cab7452c3f4d14d1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:03.102839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-674cb488-993a-40b6-ada4-4a94b80f33e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:03.227737 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1020/2071] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:03.286942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b30f6-e76c-4c84-b9f0-505f2265aa98 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:03.289854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-323b30f6-e76c-4c84-b9f0-505f2265aa98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:03.421463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b30f6-e76c-4c84-b9f0-505f2265aa98 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:03.431588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57d3c-3e43-4335-a29f-f1d7fea20a12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:03.432646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57d3c-3e43-4335-a29f-f1d7fea20a12 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:03.433100 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57d3c-3e43-4335-a29f-f1d7fea20a12 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:03.433560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57d3c-3e43-4335-a29f-f1d7fea20a12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:03.462451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e57d3c-3e43-4335-a29f-f1d7fea20a12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:03.465357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e57d3c-3e43-4335-a29f-f1d7fea20a12 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:03.488811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1052/2072] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:03.731379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cb488-993a-40b6-ada4-4a94b80f33e2 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:03.733396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-674cb488-993a-40b6-ada4-4a94b80f33e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:03.748294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cb488-993a-40b6-ada4-4a94b80f33e2 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:03.759878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a99b43-da64-4dd0-9bdd-2807d0f9dbce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:03.761103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a99b43-da64-4dd0-9bdd-2807d0f9dbce service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:03.761645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a99b43-da64-4dd0-9bdd-2807d0f9dbce service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:03.762012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a99b43-da64-4dd0-9bdd-2807d0f9dbce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:03.871887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a99b43-da64-4dd0-9bdd-2807d0f9dbce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:03.874392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a99b43-da64-4dd0-9bdd-2807d0f9dbce service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:03.887101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1021/2073] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:03.896472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:04.422169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:04.538767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57d3c-3e43-4335-a29f-f1d7fea20a12 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:04.540732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-27e57d3c-3e43-4335-a29f-f1d7fea20a12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:04.672924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57d3c-3e43-4335-a29f-f1d7fea20a12 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:04.695148 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:04.698736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:04.699584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:04.700276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] PATH_INFO: `/v3/users/c9a857096e2e4a659697559ea410bcbe` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:04.702677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a857096e2e4a659697559ea410bcbe)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:04.709399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:05.082798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a857096e2e4a659697559ea410bcbe'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:05.084220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a857096e2e4a659697559ea410bcbe'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:05.101657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a857096e2e4a659697559ea410bcbe'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:05.103200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a857096e2e4a659697559ea410bcbe'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:05.259807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a857096e2e4a659697559ea410bcbe'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:05.299503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1053/2074] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:04 2022] DELETE /identity/v3/users/c9a857096e2e4a659697559ea410bcbe => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:05.300612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc11a1-eab1-4de5-a793-d675e7509a5d None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:05.301386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc11a1-eab1-4de5-a793-d675e7509a5d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:05.301386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc11a1-eab1-4de5-a793-d675e7509a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:05.348873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-97cc11a1-eab1-4de5-a793-d675e7509a5d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:05.353273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-97cc11a1-eab1-4de5-a793-d675e7509a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:05.548607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1022/2075] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:04:05.569028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:05.571566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:05.701338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673e1f2-c578-4876-a726-9b44a0519640 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:05.709573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f3e9d-7725-4ebc-bf66-73cf5137f30e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:05.711390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f3e9d-7725-4ebc-bf66-73cf5137f30e service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:05.712200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f3e9d-7725-4ebc-bf66-73cf5137f30e service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:05.712901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f3e9d-7725-4ebc-bf66-73cf5137f30e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:05.737495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f3e9d-7725-4ebc-bf66-73cf5137f30e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:05.739362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f3e9d-7725-4ebc-bf66-73cf5137f30e service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:05.762326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1054/2076] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:06.677065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc11a1-eab1-4de5-a793-d675e7509a5d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:06.680370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-97cc11a1-eab1-4de5-a793-d675e7509a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:06.702611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc11a1-eab1-4de5-a793-d675e7509a5d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:06.712584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:06.712584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:06.712584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:06.712584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] PATH_INFO: `/v3/projects/61e730edaa844158b1d1a1470fc27d10` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:06.713171 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=61e730edaa844158b1d1a1470fc27d10)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:06.716388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:06.848943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:06.964553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61e730edaa844158b1d1a1470fc27d10', 'request_id': 'req-a0849987-39ca-48dd-a814-4479ffafbdcf'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:06.984654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61e730edaa844158b1d1a1470fc27d10', 'request_id': 'req-a0849987-39ca-48dd-a814-4479ffafbdcf'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.023416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61e730edaa844158b1d1a1470fc27d10', 'request_id': 'req-a0849987-39ca-48dd-a814-4479ffafbdcf'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.187066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] The token cache is being invalidate because project 61e730edaa844158b1d1a1470fc27d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:07.187363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.189227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.190124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1023/2077] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:06 2022] DELETE /identity/v3/projects/61e730edaa844158b1d1a1470fc27d10 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:07.194500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f3e9d-7725-4ebc-bf66-73cf5137f30e service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:07.207142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f3e9d-7725-4ebc-bf66-73cf5137f30e service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:07.303252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:07.310923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:07.310923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:07.310923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:07.310923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] PATH_INFO: `/v3/users/b1f66c0f272a4157bab384d30786dd73` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:07.311479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:07.311716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1f66c0f272a4157bab384d30786dd73)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:07.313652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:07.328086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0849987-39ca-48dd-a814-4479ffafbdcf admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:07.338675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:07.339610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:07.340160 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:07.340620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] PATH_INFO: `/v3/users/d2cbf52a4f714ad292b5088ab7c4d52d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:07.342925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2cbf52a4f714ad292b5088ab7c4d52d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:07.346738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:07.405190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:07.630817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2cbf52a4f714ad292b5088ab7c4d52d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.649442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2cbf52a4f714ad292b5088ab7c4d52d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.651483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2cbf52a4f714ad292b5088ab7c4d52d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.662449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1f66c0f272a4157bab384d30786dd73'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.666137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1f66c0f272a4157bab384d30786dd73'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.680251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1f66c0f272a4157bab384d30786dd73'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.682460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1f66c0f272a4157bab384d30786dd73'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.769029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2cbf52a4f714ad292b5088ab7c4d52d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.790570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-a0ef3595-e3f5-4502-9659-3e561433e97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2cbf52a4f714ad292b5088ab7c4d52d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.804893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1024/2078] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:07 2022] DELETE /identity/v3/users/d2cbf52a4f714ad292b5088ab7c4d52d => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:07.808500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c994dcc-2de9-4489-a982-2ce6d628d968 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:07.809122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c994dcc-2de9-4489-a982-2ce6d628d968 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:07.809855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c994dcc-2de9-4489-a982-2ce6d628d968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:07.841220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-8c994dcc-2de9-4489-a982-2ce6d628d968 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:07.842661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8c994dcc-2de9-4489-a982-2ce6d628d968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:07.888142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1f66c0f272a4157bab384d30786dd73'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:07.945391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1055/2079] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:07 2022] DELETE /identity/v3/users/b1f66c0f272a4157bab384d30786dd73 => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:07.976315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1025/2080] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:07.992667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:07.995812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:08.163033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f6aeb-76cc-403e-8c88-293b84a43e2c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:08.169398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf5bf0-52ce-4baa-a5f9-b744dae97663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:08.170239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf5bf0-52ce-4baa-a5f9-b744dae97663 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:08.170613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf5bf0-52ce-4baa-a5f9-b744dae97663 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:08.171007 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf5bf0-52ce-4baa-a5f9-b744dae97663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:08.306529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf5bf0-52ce-4baa-a5f9-b744dae97663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:08.308142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf5bf0-52ce-4baa-a5f9-b744dae97663 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:08.334351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1056/2081] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:08.515802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c994dcc-2de9-4489-a982-2ce6d628d968 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:08.517360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8c994dcc-2de9-4489-a982-2ce6d628d968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:08.531649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c994dcc-2de9-4489-a982-2ce6d628d968 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:08.537972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:08.539011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:08.539458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:08.540121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] PATH_INFO: `/v3/users/8e9f17e2252c4d04b672454aa550dafe` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:08.545063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e9f17e2252c4d04b672454aa550dafe)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:08.547166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:08.704671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf5bf0-52ce-4baa-a5f9-b744dae97663 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:08.706405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-60cf5bf0-52ce-4baa-a5f9-b744dae97663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:08.731775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf5bf0-52ce-4baa-a5f9-b744dae97663 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:08.747920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:08.747920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:08.747920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:08.747920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] PATH_INFO: `/v3/projects/8250850dcdef46708643dac707e210d7` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:08.749242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8250850dcdef46708643dac707e210d7)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:08.751716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:08.831775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e9f17e2252c4d04b672454aa550dafe'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:08.847424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e9f17e2252c4d04b672454aa550dafe'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:08.848639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e9f17e2252c4d04b672454aa550dafe'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:08.991859 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e9f17e2252c4d04b672454aa550dafe'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.004455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8250850dcdef46708643dac707e210d7', 'request_id': 'req-05cb11b6-92fd-4784-9136-4fd37f03b2f9'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.013204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8250850dcdef46708643dac707e210d7', 'request_id': 'req-05cb11b6-92fd-4784-9136-4fd37f03b2f9'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.013479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b7cc4555-f1c7-487a-aafc-7ddb367b9771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e9f17e2252c4d04b672454aa550dafe'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.023059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1026/2082] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:08 2022] DELETE /identity/v3/users/8e9f17e2252c4d04b672454aa550dafe => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:09.025210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ece0e-ea62-4191-b422-3f1ff72251c3 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:09.025552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ece0e-ea62-4191-b422-3f1ff72251c3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:09.025835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ece0e-ea62-4191-b422-3f1ff72251c3 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:09.027294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1027/2083] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:04:09.028600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620dfb9-343b-41d6-ab74-801a1d56e2d8 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:09.028976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620dfb9-343b-41d6-ab74-801a1d56e2d8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:09.029290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620dfb9-343b-41d6-ab74-801a1d56e2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:09.051530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-9620dfb9-343b-41d6-ab74-801a1d56e2d8 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:09.053100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-9620dfb9-343b-41d6-ab74-801a1d56e2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:09.125363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1028/2084] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:09.127284 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dab354-6fe5-463a-b87a-9282c4010a97 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:09.128026 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dab354-6fe5-463a-b87a-9282c4010a97 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:09.128567 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dab354-6fe5-463a-b87a-9282c4010a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:09.152457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8250850dcdef46708643dac707e210d7', 'request_id': 'req-05cb11b6-92fd-4784-9136-4fd37f03b2f9'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.164131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f0dab354-6fe5-463a-b87a-9282c4010a97 None None] MFA Rules not processed for user `16374dbb6d084ddfa3b49afd19c7c325`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:09.165965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f0dab354-6fe5-463a-b87a-9282c4010a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:09.202454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] The token cache is being invalidate because project 8250850dcdef46708643dac707e210d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:09.202454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.202454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.202454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1057/2085] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:08 2022] DELETE /identity/v3/projects/8250850dcdef46708643dac707e210d7 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:09.202454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:09.211993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:09.273601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:09.332063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1029/2086] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:04:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:09.333885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dab354-6fe5-463a-b87a-9282c4010a97 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:09.342158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f0dab354-6fe5-463a-b87a-9282c4010a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:09.368441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:09.379875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dab354-6fe5-463a-b87a-9282c4010a97 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:09.386174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb11b6-92fd-4784-9136-4fd37f03b2f9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:09.396932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:09.398496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:09.401725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:09.401725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] PATH_INFO: `/v3/users/64e03bf24bb44add8adea800932e2f75` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:09.405622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e03bf24bb44add8adea800932e2f75)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:09.409901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:09.465700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c17f9-618a-493a-93b4-e3212f8c4312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:09.466562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c17f9-618a-493a-93b4-e3212f8c4312 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:09.466971 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c17f9-618a-493a-93b4-e3212f8c4312 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:09.467499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c17f9-618a-493a-93b4-e3212f8c4312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:09.471887 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-bb9c17f9-618a-493a-93b4-e3212f8c4312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:09.490787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c17f9-618a-493a-93b4-e3212f8c4312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:09.494551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c17f9-618a-493a-93b4-e3212f8c4312 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:09.515085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1058/2087] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:09.517880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c17f9-618a-493a-93b4-e3212f8c4312 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:09.522325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-bb9c17f9-618a-493a-93b4-e3212f8c4312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:09.542264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c17f9-618a-493a-93b4-e3212f8c4312 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:09.549613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:09.550731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:09.550731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:09.551090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] PATH_INFO: `/v3/users/b1efbce16c9d45cb93580dfa0798e32f` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:09.552615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1efbce16c9d45cb93580dfa0798e32f)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:09.555383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:09.737467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e03bf24bb44add8adea800932e2f75'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.757312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e03bf24bb44add8adea800932e2f75'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.758518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e03bf24bb44add8adea800932e2f75'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.883919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e03bf24bb44add8adea800932e2f75'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.909670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1efbce16c9d45cb93580dfa0798e32f'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.910365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-3923b74f-6c40-4618-8a6d-de2f62067406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e03bf24bb44add8adea800932e2f75'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.910868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1efbce16c9d45cb93580dfa0798e32f'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.925130 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1efbce16c9d45cb93580dfa0798e32f'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.925961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1030/2088] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:09 2022] DELETE /identity/v3/users/64e03bf24bb44add8adea800932e2f75 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:09.929076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1efbce16c9d45cb93580dfa0798e32f'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:09.929258 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161c08e-b158-4453-82a5-18618bc8fa82 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:09.929258 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161c08e-b158-4453-82a5-18618bc8fa82 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:09.929516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161c08e-b158-4453-82a5-18618bc8fa82 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:09.942328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1031/2089] 10.222.0.15 () {54 vars in 961 bytes} [Tue Oct 4 12:04:09 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:04:09.944286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161c08e-b158-4453-82a5-18618bc8fa82 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:09.947689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7161c08e-b158-4453-82a5-18618bc8fa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:10.066632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8e9d2793-79e3-4b80-a3e5-dafb0f359574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1efbce16c9d45cb93580dfa0798e32f'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:10.094882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161c08e-b158-4453-82a5-18618bc8fa82 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:10.106109 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:10.107157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:10.107636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:10.108147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] PATH_INFO: `/v3/projects/1adf43ebb9b84aec9beb89d36fd48eb3` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:10.109871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1adf43ebb9b84aec9beb89d36fd48eb3)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:10.114312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:10.114883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1059/2090] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:09 2022] DELETE /identity/v3/users/b1efbce16c9d45cb93580dfa0798e32f => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:10.116824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d2645-4450-4420-98b4-7d7b5c42d0b2 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:10.117320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d2645-4450-4420-98b4-7d7b5c42d0b2 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:10.133292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d2645-4450-4420-98b4-7d7b5c42d0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:10.206125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c64d2645-4450-4420-98b4-7d7b5c42d0b2 None None] MFA Rules not processed for user `16374dbb6d084ddfa3b49afd19c7c325`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:10.212090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c64d2645-4450-4420-98b4-7d7b5c42d0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:10.310915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1adf43ebb9b84aec9beb89d36fd48eb3', 'request_id': 'req-ade57f94-deb6-4768-8b99-afad296f1a7b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:10.327665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1adf43ebb9b84aec9beb89d36fd48eb3', 'request_id': 'req-ade57f94-deb6-4768-8b99-afad296f1a7b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:10.350390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1adf43ebb9b84aec9beb89d36fd48eb3', 'request_id': 'req-ade57f94-deb6-4768-8b99-afad296f1a7b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:10.406922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1060/2091] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Oct 4 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:10.420931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d2645-4450-4420-98b4-7d7b5c42d0b2 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:10.439820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d2645-4450-4420-98b4-7d7b5c42d0b2 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:10.484439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] The token cache is being invalidate because project 1adf43ebb9b84aec9beb89d36fd48eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:10.484439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:10.485680 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ade57f94-deb6-4768-8b99-afad296f1a7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:10.487187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1032/2092] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:09 2022] DELETE /identity/v3/projects/1adf43ebb9b84aec9beb89d36fd48eb3 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:10.493426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb750c6d-36e4-4166-82c3-5118dd9601dc None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:10.494168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb750c6d-36e4-4166-82c3-5118dd9601dc None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:10.494168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb750c6d-36e4-4166-82c3-5118dd9601dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:10.517586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-bb750c6d-36e4-4166-82c3-5118dd9601dc None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:10.518786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-bb750c6d-36e4-4166-82c3-5118dd9601dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:10.607615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd03d3-b48e-458e-b38f-7778983f69dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:10.609015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd03d3-b48e-458e-b38f-7778983f69dc service placement] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:10.609595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd03d3-b48e-458e-b38f-7778983f69dc service placement] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:10.610018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd03d3-b48e-458e-b38f-7778983f69dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:10.612857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-81bd03d3-b48e-458e-b38f-7778983f69dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:10.618621 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1033/2093] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:10.621904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb750c6d-36e4-4166-82c3-5118dd9601dc None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:10.629216 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-bb750c6d-36e4-4166-82c3-5118dd9601dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:10.632563 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd03d3-b48e-458e-b38f-7778983f69dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:10.635573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd03d3-b48e-458e-b38f-7778983f69dc service placement] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:10.653387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb750c6d-36e4-4166-82c3-5118dd9601dc None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:10.653585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1061/2094] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:04:10.654554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2ac1-3c2e-4d7d-a3ed-98da5b36e42e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:10.654922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2ac1-3c2e-4d7d-a3ed-98da5b36e42e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:10.655208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f2ac1-3c2e-4d7d-a3ed-98da5b36e42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:10.674856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:10.680777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9d6f2ac1-3c2e-4d7d-a3ed-98da5b36e42e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:10.682218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:10.682384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9d6f2ac1-3c2e-4d7d-a3ed-98da5b36e42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:10.683075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:10.683773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] PATH_INFO: `/v3/projects/e83502415eb6488c9ea4315ef49a4e9a` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:10.687236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e83502415eb6488c9ea4315ef49a4e9a)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:10.692790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:10.697461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1062/2095] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:10.698678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2ac1-3c2e-4d7d-a3ed-98da5b36e42e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:10.711685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f2ac1-3c2e-4d7d-a3ed-98da5b36e42e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:10.719900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d26e1-4ef9-4a28-8888-0cb40081cbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:10.720729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d26e1-4ef9-4a28-8888-0cb40081cbde admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:10.721352 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d26e1-4ef9-4a28-8888-0cb40081cbde admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:10.721732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d26e1-4ef9-4a28-8888-0cb40081cbde admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:10.722671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d26e1-4ef9-4a28-8888-0cb40081cbde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:10.723024 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d26e1-4ef9-4a28-8888-0cb40081cbde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:10.725113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d26e1-4ef9-4a28-8888-0cb40081cbde admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:10.752647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1063/2096] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:10.752647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d26e1-4ef9-4a28-8888-0cb40081cbde admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:10.752647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cb5d26e1-4ef9-4a28-8888-0cb40081cbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:10.763704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d26e1-4ef9-4a28-8888-0cb40081cbde admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:10.777164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:10.779255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:10.779255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:10.779255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] PATH_INFO: `/v3/projects/b32ff677e86d4be0b49869e579385a21` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:10.779712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32ff677e86d4be0b49869e579385a21)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:10.781732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:10.897791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e83502415eb6488c9ea4315ef49a4e9a', 'request_id': 'req-06797252-3086-4641-9ad1-ffaa61e00e37'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:10.904911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e83502415eb6488c9ea4315ef49a4e9a', 'request_id': 'req-06797252-3086-4641-9ad1-ffaa61e00e37'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:10.913185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e83502415eb6488c9ea4315ef49a4e9a', 'request_id': 'req-06797252-3086-4641-9ad1-ffaa61e00e37'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.006196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32ff677e86d4be0b49869e579385a21', 'request_id': 'req-fe05c5d1-9dad-4609-8ea1-9743bc41a162'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.012808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32ff677e86d4be0b49869e579385a21', 'request_id': 'req-fe05c5d1-9dad-4609-8ea1-9743bc41a162'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.049482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] The token cache is being invalidate because project e83502415eb6488c9ea4315ef49a4e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:11.049847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.050646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.052383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1034/2097] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:10 2022] DELETE /identity/v3/projects/e83502415eb6488c9ea4315ef49a4e9a => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:11.053507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:11.055122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:11.133756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32ff677e86d4be0b49869e579385a21', 'request_id': 'req-fe05c5d1-9dad-4609-8ea1-9743bc41a162'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.144694 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06797252-3086-4641-9ad1-ffaa61e00e37 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:11.148657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] The token cache is being invalidate because project b32ff677e86d4be0b49869e579385a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:11.149143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.149726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.151547 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1064/2098] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:10 2022] DELETE /identity/v3/projects/b32ff677e86d4be0b49869e579385a21 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:11.152134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:11.154347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:11.223921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c961d-a67c-43c0-bee7-e838bd5ddaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:11.224648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c961d-a67c-43c0-bee7-e838bd5ddaed admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:11.224952 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c961d-a67c-43c0-bee7-e838bd5ddaed admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:11.225253 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c961d-a67c-43c0-bee7-e838bd5ddaed admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:11.225854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c961d-a67c-43c0-bee7-e838bd5ddaed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:11.227854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c961d-a67c-43c0-bee7-e838bd5ddaed admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:11.261268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05c5d1-9dad-4609-8ea1-9743bc41a162 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:11.267849 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:11.268721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:11.269084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:11.269465 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] PATH_INFO: `/v3/projects/1728825365384205b0487c3c46ced621` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:11.271174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1728825365384205b0487c3c46ced621)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:11.273271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:11.431767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1035/2099] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:11 2022] POST /identity/v3/projects => generated 371 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:11.433781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c961d-a67c-43c0-bee7-e838bd5ddaed admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:11.437169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-813c961d-a67c-43c0-bee7-e838bd5ddaed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:11.444583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1728825365384205b0487c3c46ced621', 'request_id': 'req-471f94d0-16eb-4866-a030-4857bf8e0759'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.450920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:11.452869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1728825365384205b0487c3c46ced621', 'request_id': 'req-471f94d0-16eb-4866-a030-4857bf8e0759'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.545657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c961d-a67c-43c0-bee7-e838bd5ddaed admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:11.553068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:11.554584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:11.555342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:11.556283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] PATH_INFO: `/v3/users/f05ebe8158f441cab7452c3f4d14d1d6` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:11.558650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05ebe8158f441cab7452c3f4d14d1d6)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:11.562252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:11.566446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1728825365384205b0487c3c46ced621', 'request_id': 'req-471f94d0-16eb-4866-a030-4857bf8e0759'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.587444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] The token cache is being invalidate because project 1728825365384205b0487c3c46ced621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:11.588415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.589260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-471f94d0-16eb-4866-a030-4857bf8e0759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:11.591551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1065/2100] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:11 2022] DELETE /identity/v3/projects/1728825365384205b0487c3c46ced621 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:11.598322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d123dc-1c31-41ea-b384-32109c346f41 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:11.598777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d123dc-1c31-41ea-b384-32109c346f41 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:11.599810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d123dc-1c31-41ea-b384-32109c346f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:11.629188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c8d123dc-1c31-41ea-b384-32109c346f41 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:11.630707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c8d123dc-1c31-41ea-b384-32109c346f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:11.640784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1066/2101] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:11.642635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d123dc-1c31-41ea-b384-32109c346f41 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:11.644252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c8d123dc-1c31-41ea-b384-32109c346f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:11.657829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d123dc-1c31-41ea-b384-32109c346f41 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:11.663001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e9b37-a2e7-4ecb-b569-247e8ff794d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:11.663626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e9b37-a2e7-4ecb-b569-247e8ff794d1 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:11.663913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e9b37-a2e7-4ecb-b569-247e8ff794d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:11.664206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e9b37-a2e7-4ecb-b569-247e8ff794d1 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:11.664880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e9b37-a2e7-4ecb-b569-247e8ff794d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:11.666836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e9b37-a2e7-4ecb-b569-247e8ff794d1 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:11.956736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1067/2102] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:11 2022] POST /identity/v3/users => generated 286 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:11.956736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425d6f4-a5e6-4b24-beee-0e7ac2bd4453 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:11.956736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425d6f4-a5e6-4b24-beee-0e7ac2bd4453 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:11.956736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425d6f4-a5e6-4b24-beee-0e7ac2bd4453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:11.982933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2425d6f4-a5e6-4b24-beee-0e7ac2bd4453 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:11.984362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2425d6f4-a5e6-4b24-beee-0e7ac2bd4453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:11.993936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1068/2103] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:11.995376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95b23f-2844-436e-a4db-66357969b8a0 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:11.995692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95b23f-2844-436e-a4db-66357969b8a0 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:11.996042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95b23f-2844-436e-a4db-66357969b8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.021211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-ce95b23f-2844-436e-a4db-66357969b8a0 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:12.022529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ce95b23f-2844-436e-a4db-66357969b8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:12.031562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1069/2104] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:12.032539 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95b23f-2844-436e-a4db-66357969b8a0 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:12.044860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95b23f-2844-436e-a4db-66357969b8a0 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:12.051499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbe684-fc9c-416d-9112-3f96f9dd5b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:12.052257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbe684-fc9c-416d-9112-3f96f9dd5b7f admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.052422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbe684-fc9c-416d-9112-3f96f9dd5b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.052729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbe684-fc9c-416d-9112-3f96f9dd5b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.053375 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbe684-fc9c-416d-9112-3f96f9dd5b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:12.055049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbe684-fc9c-416d-9112-3f96f9dd5b7f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:12.058528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05ebe8158f441cab7452c3f4d14d1d6'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:12.067303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1070/2105] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:12.067303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1a370-4229-47d2-9e27-0ea23d1b5dee None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.067303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1a370-4229-47d2-9e27-0ea23d1b5dee None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.067303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1a370-4229-47d2-9e27-0ea23d1b5dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.077859 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05ebe8158f441cab7452c3f4d14d1d6'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:12.080262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05ebe8158f441cab7452c3f4d14d1d6'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:12.094712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2bd1a370-4229-47d2-9e27-0ea23d1b5dee None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:12.095953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2bd1a370-4229-47d2-9e27-0ea23d1b5dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:12.107159 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1071/2106] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:12.108511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1a370-4229-47d2-9e27-0ea23d1b5dee None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:12.111790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2bd1a370-4229-47d2-9e27-0ea23d1b5dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:12.126673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1a370-4229-47d2-9e27-0ea23d1b5dee None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:12.132223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568fcc2-2de3-448e-a72b-85f603c61dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:12.132892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568fcc2-2de3-448e-a72b-85f603c61dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.133171 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568fcc2-2de3-448e-a72b-85f603c61dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.133449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568fcc2-2de3-448e-a72b-85f603c61dc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.134121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568fcc2-2de3-448e-a72b-85f603c61dc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:12.134429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568fcc2-2de3-448e-a72b-85f603c61dc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:12.136104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568fcc2-2de3-448e-a72b-85f603c61dc9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:12.154459 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1072/2107] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:12.155338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568fcc2-2de3-448e-a72b-85f603c61dc9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:12.157408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1568fcc2-2de3-448e-a72b-85f603c61dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:12.170821 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568fcc2-2de3-448e-a72b-85f603c61dc9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:12.176506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89100271-898b-4d7e-b47d-d0d48999d2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:12.177129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89100271-898b-4d7e-b47d-d0d48999d2ec admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.177412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89100271-898b-4d7e-b47d-d0d48999d2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.177694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89100271-898b-4d7e-b47d-d0d48999d2ec admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.178352 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89100271-898b-4d7e-b47d-d0d48999d2ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:12.180000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89100271-898b-4d7e-b47d-d0d48999d2ec admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:12.192022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1073/2108] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:12.193031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89100271-898b-4d7e-b47d-d0d48999d2ec admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:12.205053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89100271-898b-4d7e-b47d-d0d48999d2ec admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:12.210773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa2b4e-89fc-4f97-b7b1-a8e8e07a739c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:12.211540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa2b4e-89fc-4f97-b7b1-a8e8e07a739c admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.211920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa2b4e-89fc-4f97-b7b1-a8e8e07a739c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.212462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa2b4e-89fc-4f97-b7b1-a8e8e07a739c admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.213378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa2b4e-89fc-4f97-b7b1-a8e8e07a739c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:12.213854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa2b4e-89fc-4f97-b7b1-a8e8e07a739c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:12.215912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa2b4e-89fc-4f97-b7b1-a8e8e07a739c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:12.221725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:12.232899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1074/2109] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:12.234018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa2b4e-89fc-4f97-b7b1-a8e8e07a739c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:12.245606 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa2b4e-89fc-4f97-b7b1-a8e8e07a739c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:12.250976 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6c14e-ae53-4c40-8085-f28f95ff1a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:12.251611 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6c14e-ae53-4c40-8085-f28f95ff1a98 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.251894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6c14e-ae53-4c40-8085-f28f95ff1a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.252207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6c14e-ae53-4c40-8085-f28f95ff1a98 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.253179 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6c14e-ae53-4c40-8085-f28f95ff1a98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:12.255198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6c14e-ae53-4c40-8085-f28f95ff1a98 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:12.259001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05ebe8158f441cab7452c3f4d14d1d6'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:12.279559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05ebe8158f441cab7452c3f4d14d1d6'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:12.288116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1036/2110] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:11 2022] DELETE /identity/v3/users/f05ebe8158f441cab7452c3f4d14d1d6 => generated 0 bytes in 856 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:12.289325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:12.408558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e7a4d-366a-423c-9785-d78a78b0d6ce admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:12.417396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e468d7-a7c4-40d5-b0d7-6952eaca1648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:12.418749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e468d7-a7c4-40d5-b0d7-6952eaca1648 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.419255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e468d7-a7c4-40d5-b0d7-6952eaca1648 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.420296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e468d7-a7c4-40d5-b0d7-6952eaca1648 admin admin] PATH_INFO: `/v3/projects/df93e841cba9441f84c15a826db25de6/users/ee6a9052a75a4d8c9f4a08638685012b/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.449612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1075/2111] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:12 2022] POST /identity/v3/projects => generated 391 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:12.452115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e468d7-a7c4-40d5-b0d7-6952eaca1648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df93e841cba9441f84c15a826db25de6, user_id=ee6a9052a75a4d8c9f4a08638685012b, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:12.457161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e468d7-a7c4-40d5-b0d7-6952eaca1648 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:12.458574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6c14e-ae53-4c40-8085-f28f95ff1a98 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:12.559088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6c14e-ae53-4c40-8085-f28f95ff1a98 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:12.565949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab9eb3-c493-4627-b7fa-1b695633a852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:12.566785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9eb3-c493-4627-b7fa-1b695633a852 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.567156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9eb3-c493-4627-b7fa-1b695633a852 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.567744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9eb3-c493-4627-b7fa-1b695633a852 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.568959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab9eb3-c493-4627-b7fa-1b695633a852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:12.571207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab9eb3-c493-4627-b7fa-1b695633a852 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:12.694573 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1037/2112] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:12 2022] PUT /identity/v3/projects/df93e841cba9441f84c15a826db25de6/users/ee6a9052a75a4d8c9f4a08638685012b/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:12.695742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e468d7-a7c4-40d5-b0d7-6952eaca1648 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:12.768075 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1076/2113] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:12 2022] POST /identity/v3/projects => generated 389 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:12.769809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c13f-c493-4b6f-9202-98d0177b3489 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.770351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c13f-c493-4b6f-9202-98d0177b3489 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.770633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c13f-c493-4b6f-9202-98d0177b3489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.831411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e468d7-a7c4-40d5-b0d7-6952eaca1648 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:12.831858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-a2b1c13f-c493-4b6f-9202-98d0177b3489 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:12.833963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a2b1c13f-c493-4b6f-9202-98d0177b3489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:12.937582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df857669-f47b-4dce-95d2-f3d563050c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:12.939444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df857669-f47b-4dce-95d2-f3d563050c0f admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.939876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df857669-f47b-4dce-95d2-f3d563050c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.940356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df857669-f47b-4dce-95d2-f3d563050c0f admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.941790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df857669-f47b-4dce-95d2-f3d563050c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:12.945932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df857669-f47b-4dce-95d2-f3d563050c0f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:12.961966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1077/2114] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:12.961966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c13f-c493-4b6f-9202-98d0177b3489 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:12.981876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c13f-c493-4b6f-9202-98d0177b3489 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:12.991293 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b31562-9fcc-45b5-83c1-2e560b10f308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:12.992225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b31562-9fcc-45b5-83c1-2e560b10f308 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:12.992642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b31562-9fcc-45b5-83c1-2e560b10f308 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:12.993098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b31562-9fcc-45b5-83c1-2e560b10f308 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:12.994171 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b31562-9fcc-45b5-83c1-2e560b10f308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:12.997046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b31562-9fcc-45b5-83c1-2e560b10f308 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:13.195409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1038/2115] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:12 2022] POST /identity/v3/users => generated 296 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:13.200067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df857669-f47b-4dce-95d2-f3d563050c0f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:13.200377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-df857669-f47b-4dce-95d2-f3d563050c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:13.238172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1078/2116] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:12 2022] POST /identity/v3/users => generated 295 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:13.240903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b31562-9fcc-45b5-83c1-2e560b10f308 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:13.284461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b31562-9fcc-45b5-83c1-2e560b10f308 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:13.295773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afadc0-cd1f-4928-83fb-50a582efcfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:13.297289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afadc0-cd1f-4928-83fb-50a582efcfa9 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:13.297911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afadc0-cd1f-4928-83fb-50a582efcfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:13.298413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afadc0-cd1f-4928-83fb-50a582efcfa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:13.300216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afadc0-cd1f-4928-83fb-50a582efcfa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:13.303837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afadc0-cd1f-4928-83fb-50a582efcfa9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:13.324752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df857669-f47b-4dce-95d2-f3d563050c0f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:13.326176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1079/2117] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:13.327590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afadc0-cd1f-4928-83fb-50a582efcfa9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:13.333131 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d4afadc0-cd1f-4928-83fb-50a582efcfa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:13.334360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95537cbe-98fb-44bd-a298-6628ac4295eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:13.335279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95537cbe-98fb-44bd-a298-6628ac4295eb service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:13.335764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95537cbe-98fb-44bd-a298-6628ac4295eb service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:13.336311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95537cbe-98fb-44bd-a298-6628ac4295eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:13.350350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95537cbe-98fb-44bd-a298-6628ac4295eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:13.350635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afadc0-cd1f-4928-83fb-50a582efcfa9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:13.352280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95537cbe-98fb-44bd-a298-6628ac4295eb service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:13.358388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:13.359546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:13.360489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:13.360983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] PATH_INFO: `/v3/projects/de6db739a16648b2bfd99977f99f96e3` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:13.362950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de6db739a16648b2bfd99977f99f96e3)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:13.366957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:13.371082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1039/2118] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:13.372670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95537cbe-98fb-44bd-a298-6628ac4295eb service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:13.389659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95537cbe-98fb-44bd-a298-6628ac4295eb service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:13.399724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820173a3-048b-40c7-b2ef-297383489414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:13.400781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820173a3-048b-40c7-b2ef-297383489414 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:13.401319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820173a3-048b-40c7-b2ef-297383489414 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:13.402133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820173a3-048b-40c7-b2ef-297383489414 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:13.403124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820173a3-048b-40c7-b2ef-297383489414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:13.405955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820173a3-048b-40c7-b2ef-297383489414 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:13.422806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1040/2119] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:13.425338 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820173a3-048b-40c7-b2ef-297383489414 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:13.437990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820173a3-048b-40c7-b2ef-297383489414 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:13.443707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a634407-3451-4116-8335-3ff453821b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:13.444333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a634407-3451-4116-8335-3ff453821b72 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:13.444729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a634407-3451-4116-8335-3ff453821b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:13.445119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a634407-3451-4116-8335-3ff453821b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:13.445938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a634407-3451-4116-8335-3ff453821b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:13.447862 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a634407-3451-4116-8335-3ff453821b72 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:13.460285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1041/2120] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:13.461191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a634407-3451-4116-8335-3ff453821b72 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:13.472826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a634407-3451-4116-8335-3ff453821b72 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:13.477991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fe43e-1dec-4c90-be9e-495b3dbad451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:13.478620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fe43e-1dec-4c90-be9e-495b3dbad451 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:13.478899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fe43e-1dec-4c90-be9e-495b3dbad451 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:13.479173 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fe43e-1dec-4c90-be9e-495b3dbad451 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:13.479856 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fe43e-1dec-4c90-be9e-495b3dbad451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:13.481562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fe43e-1dec-4c90-be9e-495b3dbad451 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:13.494474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:13.499209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1042/2121] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:13.500574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fe43e-1dec-4c90-be9e-495b3dbad451 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:13.512258 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fe43e-1dec-4c90-be9e-495b3dbad451 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:13.517981 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d53968-2bd1-4166-83a0-acb2d532035c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:13.518688 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53968-2bd1-4166-83a0-acb2d532035c admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:13.518960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53968-2bd1-4166-83a0-acb2d532035c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:13.519248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53968-2bd1-4166-83a0-acb2d532035c admin admin] PATH_INFO: `/v3/projects/f4f180049cc049829e12eace469bef05/users/f3f607e48e534dd09c3619982978e9e5/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:13.538508 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d53968-2bd1-4166-83a0-acb2d532035c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f180049cc049829e12eace469bef05, user_id=f3f607e48e534dd09c3619982978e9e5, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:13.541354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d53968-2bd1-4166-83a0-acb2d532035c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:13.558217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de6db739a16648b2bfd99977f99f96e3', 'request_id': 'req-25adfbba-2a54-4182-8e4e-31d64e2bd506'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:13.564919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de6db739a16648b2bfd99977f99f96e3', 'request_id': 'req-25adfbba-2a54-4182-8e4e-31d64e2bd506'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:13.729523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de6db739a16648b2bfd99977f99f96e3', 'request_id': 'req-25adfbba-2a54-4182-8e4e-31d64e2bd506'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:13.738487 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1043/2122] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:13 2022] PUT /identity/v3/projects/f4f180049cc049829e12eace469bef05/users/f3f607e48e534dd09c3619982978e9e5/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:13.739426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d53968-2bd1-4166-83a0-acb2d532035c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:13.749349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] The token cache is being invalidate because project de6db739a16648b2bfd99977f99f96e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:13.749740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:13.750674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-25adfbba-2a54-4182-8e4e-31d64e2bd506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:13.752995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1080/2123] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:13 2022] DELETE /identity/v3/projects/de6db739a16648b2bfd99977f99f96e3 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:13.754955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d473fd-7d37-4ce0-9e3c-9695bd4422b2 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:13.755394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d473fd-7d37-4ce0-9e3c-9695bd4422b2 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:13.755807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d473fd-7d37-4ce0-9e3c-9695bd4422b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:13.787572 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e5d473fd-7d37-4ce0-9e3c-9695bd4422b2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:13.789604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e5d473fd-7d37-4ce0-9e3c-9695bd4422b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:13.851112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d53968-2bd1-4166-83a0-acb2d532035c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:13.862167 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482de90-74d2-46ae-9e89-122059d0179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:13.863244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482de90-74d2-46ae-9e89-122059d0179e admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:13.863773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482de90-74d2-46ae-9e89-122059d0179e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:13.864374 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482de90-74d2-46ae-9e89-122059d0179e admin admin] PATH_INFO: `/v3/projects/ba67e453721f465cad36ee25a030c759/users/f50449b73a164ad6b7fb2b01c82e7255/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:13.893148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482de90-74d2-46ae-9e89-122059d0179e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba67e453721f465cad36ee25a030c759, user_id=f50449b73a164ad6b7fb2b01c82e7255, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:13.899056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482de90-74d2-46ae-9e89-122059d0179e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:13.903934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1081/2124] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:13.906151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38629dee-f29e-46f5-b2fc-4cc7acf9a06c None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:13.906657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38629dee-f29e-46f5-b2fc-4cc7acf9a06c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:13.907956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38629dee-f29e-46f5-b2fc-4cc7acf9a06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:13.947558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-38629dee-f29e-46f5-b2fc-4cc7acf9a06c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:13.949859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-38629dee-f29e-46f5-b2fc-4cc7acf9a06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:13.965490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1082/2125] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:13.967063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38629dee-f29e-46f5-b2fc-4cc7acf9a06c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:13.969983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-38629dee-f29e-46f5-b2fc-4cc7acf9a06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:14.009326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1044/2126] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:13 2022] PUT /identity/v3/projects/ba67e453721f465cad36ee25a030c759/users/f50449b73a164ad6b7fb2b01c82e7255/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:14.009326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c5be-702a-4419-9800-9226f8dd0fb5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:14.009326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c5be-702a-4419-9800-9226f8dd0fb5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:14.009326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c5be-702a-4419-9800-9226f8dd0fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:14.053272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-2449c5be-702a-4419-9800-9226f8dd0fb5 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:14.055297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2449c5be-702a-4419-9800-9226f8dd0fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:14.075972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38629dee-f29e-46f5-b2fc-4cc7acf9a06c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:14.153262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1045/2127] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:14.155617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98342ef-a56f-4247-96da-22eaf6bf2f9a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:14.156108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98342ef-a56f-4247-96da-22eaf6bf2f9a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:14.156611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98342ef-a56f-4247-96da-22eaf6bf2f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:14.161379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e963d-6871-490e-96a2-c05d4b1001a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:14.162699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e963d-6871-490e-96a2-c05d4b1001a2 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:14.163185 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e963d-6871-490e-96a2-c05d4b1001a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:14.163635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e963d-6871-490e-96a2-c05d4b1001a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:14.177021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e963d-6871-490e-96a2-c05d4b1001a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:14.178620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e963d-6871-490e-96a2-c05d4b1001a2 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:14.191106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1083/2128] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:14.192456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e963d-6871-490e-96a2-c05d4b1001a2 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:14.202605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f98342ef-a56f-4247-96da-22eaf6bf2f9a None None] MFA Rules not processed for user `f50449b73a164ad6b7fb2b01c82e7255`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:14.206879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f98342ef-a56f-4247-96da-22eaf6bf2f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:14.230240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e963d-6871-490e-96a2-c05d4b1001a2 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:14.230240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6ce60-92e9-4750-a0bb-02995aaadf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:14.230240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6ce60-92e9-4750-a0bb-02995aaadf50 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:14.230240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6ce60-92e9-4750-a0bb-02995aaadf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:14.231295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6ce60-92e9-4750-a0bb-02995aaadf50 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:14.231295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6ce60-92e9-4750-a0bb-02995aaadf50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:14.231295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6ce60-92e9-4750-a0bb-02995aaadf50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:14.231295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6ce60-92e9-4750-a0bb-02995aaadf50 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:14.239752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1084/2129] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:14.245833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6ce60-92e9-4750-a0bb-02995aaadf50 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:14.257853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6ce60-92e9-4750-a0bb-02995aaadf50 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:14.263716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac22a84-e439-482c-8a2d-cb43ea813ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:14.264449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac22a84-e439-482c-8a2d-cb43ea813ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:14.264937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac22a84-e439-482c-8a2d-cb43ea813ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:14.265517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac22a84-e439-482c-8a2d-cb43ea813ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:14.267057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac22a84-e439-482c-8a2d-cb43ea813ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:14.270040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac22a84-e439-482c-8a2d-cb43ea813ce5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:14.318607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1046/2130] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:14.338048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98342ef-a56f-4247-96da-22eaf6bf2f9a None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:14.340605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f98342ef-a56f-4247-96da-22eaf6bf2f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:14.468458 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1085/2131] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:14 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:14.475740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac22a84-e439-482c-8a2d-cb43ea813ce5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:14.485257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98342ef-a56f-4247-96da-22eaf6bf2f9a None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:14.604923 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037f62d-1eb8-405e-a74a-d593cb97d095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:14.605569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037f62d-1eb8-405e-a74a-d593cb97d095 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:14.605855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037f62d-1eb8-405e-a74a-d593cb97d095 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:14.606146 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037f62d-1eb8-405e-a74a-d593cb97d095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:14.677518 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac22a84-e439-482c-8a2d-cb43ea813ce5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:14.683254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a26911-d8a0-4e73-805e-4b38d683dddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:14.684115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a26911-d8a0-4e73-805e-4b38d683dddb admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:14.684403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a26911-d8a0-4e73-805e-4b38d683dddb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:14.684685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a26911-d8a0-4e73-805e-4b38d683dddb admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:14.685357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a26911-d8a0-4e73-805e-4b38d683dddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:14.689230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a26911-d8a0-4e73-805e-4b38d683dddb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:14.719567 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:14.758972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037f62d-1eb8-405e-a74a-d593cb97d095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:14.765189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037f62d-1eb8-405e-a74a-d593cb97d095 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:14.803598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1086/2132] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:14 2022] POST /identity/v3/users => generated 293 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:04:14.806971 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1047/2133] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:04:14.809489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a26911-d8a0-4e73-805e-4b38d683dddb admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:14.831559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a26911-d8a0-4e73-805e-4b38d683dddb admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:14.842797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0d435-e572-44f8-8895-1a35a0eb818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:14.844365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d435-e572-44f8-8895-1a35a0eb818c admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:14.844365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d435-e572-44f8-8895-1a35a0eb818c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:14.845113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d435-e572-44f8-8895-1a35a0eb818c admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:14.846135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0d435-e572-44f8-8895-1a35a0eb818c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:14.848887 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0d435-e572-44f8-8895-1a35a0eb818c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:14.863622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1087/2134] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:14.871058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037f62d-1eb8-405e-a74a-d593cb97d095 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:14.898704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037f62d-1eb8-405e-a74a-d593cb97d095 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:14.906130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58595834-d22a-4b00-9969-d43d4c1165d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:14.911169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58595834-d22a-4b00-9969-d43d4c1165d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:14.912048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58595834-d22a-4b00-9969-d43d4c1165d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:14.912842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58595834-d22a-4b00-9969-d43d4c1165d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:14.913805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58595834-d22a-4b00-9969-d43d4c1165d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:14.916811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58595834-d22a-4b00-9969-d43d4c1165d5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:14.933001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1048/2135] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:14.939363 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0d435-e572-44f8-8895-1a35a0eb818c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:14.952278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0d435-e572-44f8-8895-1a35a0eb818c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:14.957993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b9a3c-032a-4d94-bf92-a60d49327c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:14.958788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b9a3c-032a-4d94-bf92-a60d49327c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:14.959144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b9a3c-032a-4d94-bf92-a60d49327c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:14.959607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b9a3c-032a-4d94-bf92-a60d49327c2b admin admin] PATH_INFO: `/v3/projects/7bae5c18c4e54529b2c641f1197a4717/users/d4a95eee6334412cae58ec4bdad67aad/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:14.983046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b9a3c-032a-4d94-bf92-a60d49327c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bae5c18c4e54529b2c641f1197a4717, user_id=d4a95eee6334412cae58ec4bdad67aad, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:14.986793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b9a3c-032a-4d94-bf92-a60d49327c2b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:15.154049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1088/2136] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:14 2022] PUT /identity/v3/projects/7bae5c18c4e54529b2c641f1197a4717/users/d4a95eee6334412cae58ec4bdad67aad/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:15.175834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebf207-21ca-4533-847d-cc068f3ef965 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:15.176880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebf207-21ca-4533-847d-cc068f3ef965 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:15.177680 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebf207-21ca-4533-847d-cc068f3ef965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:15.242870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-eeebf207-21ca-4533-847d-cc068f3ef965 None None] MFA Rules not processed for user `d4a95eee6334412cae58ec4bdad67aad`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:15.246883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-eeebf207-21ca-4533-847d-cc068f3ef965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:15.359966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1049/2137] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:15.366797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b9a3c-032a-4d94-bf92-a60d49327c2b admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:15.448958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b9a3c-032a-4d94-bf92-a60d49327c2b admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:15.454378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ac66f-89f0-4acb-9dd5-4b313324b992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:15.455047 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac66f-89f0-4acb-9dd5-4b313324b992 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:15.455392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac66f-89f0-4acb-9dd5-4b313324b992 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:15.455654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac66f-89f0-4acb-9dd5-4b313324b992 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:15.456324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac66f-89f0-4acb-9dd5-4b313324b992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:15.458315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac66f-89f0-4acb-9dd5-4b313324b992 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:15.600625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1089/2138] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:15 2022] POST /identity/v3/projects => generated 387 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:15.605402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebf207-21ca-4533-847d-cc068f3ef965 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:15.619073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebf207-21ca-4533-847d-cc068f3ef965 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:15.698988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36200826-3837-498c-9576-f073049e6a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:15.699907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36200826-3837-498c-9576-f073049e6a92 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:15.700382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36200826-3837-498c-9576-f073049e6a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:15.700873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36200826-3837-498c-9576-f073049e6a92 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:15.701594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36200826-3837-498c-9576-f073049e6a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:15.704974 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36200826-3837-498c-9576-f073049e6a92 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:15.815381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1050/2139] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:15 2022] POST /identity/v3/users => generated 300 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:04:15.821846 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ac66f-89f0-4acb-9dd5-4b313324b992 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:15.864437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ac66f-89f0-4acb-9dd5-4b313324b992 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:15.864437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e021bca-6f3f-4025-8579-23b445c2d46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:15.864437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021bca-6f3f-4025-8579-23b445c2d46d admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:15.864437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021bca-6f3f-4025-8579-23b445c2d46d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:15.865841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021bca-6f3f-4025-8579-23b445c2d46d admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:15.865841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e021bca-6f3f-4025-8579-23b445c2d46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:15.865841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e021bca-6f3f-4025-8579-23b445c2d46d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:15.867282 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1090/2140] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:15.872933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36200826-3837-498c-9576-f073049e6a92 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:15.894490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36200826-3837-498c-9576-f073049e6a92 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:15.923995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033ca66-f80f-4d83-af68-ad94636d364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:15.925912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ca66-f80f-4d83-af68-ad94636d364c admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:15.926637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ca66-f80f-4d83-af68-ad94636d364c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:15.927320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ca66-f80f-4d83-af68-ad94636d364c admin admin] PATH_INFO: `/v3/projects/f84ba8d7532143adbd68abf2fd9c0c42/users/5a53639476444b549a7d480d99fc571b/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:16.008377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033ca66-f80f-4d83-af68-ad94636d364c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84ba8d7532143adbd68abf2fd9c0c42, user_id=5a53639476444b549a7d480d99fc571b, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:16.016634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033ca66-f80f-4d83-af68-ad94636d364c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:16.144307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1051/2141] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:15 2022] PUT /identity/v3/projects/f84ba8d7532143adbd68abf2fd9c0c42/users/5a53639476444b549a7d480d99fc571b/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:16.150048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e021bca-6f3f-4025-8579-23b445c2d46d admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:16.164152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e021bca-6f3f-4025-8579-23b445c2d46d admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:16.216333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:16.357362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf28d-b023-4d59-a745-421741b32eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:16.358021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf28d-b023-4d59-a745-421741b32eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:16.358515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf28d-b023-4d59-a745-421741b32eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:16.358807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf28d-b023-4d59-a745-421741b32eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:16.359613 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebf28d-b023-4d59-a745-421741b32eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:16.362710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebf28d-b023-4d59-a745-421741b32eb6 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:16.383366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1091/2142] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:04:16.386241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99733c05-4c37-41db-a779-fd26d8aabb01 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:16.386737 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99733c05-4c37-41db-a779-fd26d8aabb01 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:16.386866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99733c05-4c37-41db-a779-fd26d8aabb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:16.391860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf28d-b023-4d59-a745-421741b32eb6 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:16.423882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf28d-b023-4d59-a745-421741b32eb6 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:16.431486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ca8b8-5d5a-4f2f-90cf-6c42683a05f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:16.432188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca8b8-5d5a-4f2f-90cf-6c42683a05f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:16.432512 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca8b8-5d5a-4f2f-90cf-6c42683a05f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:16.432830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca8b8-5d5a-4f2f-90cf-6c42683a05f4 admin admin] PATH_INFO: `/v3/projects/f84ba8d7532143adbd68abf2fd9c0c42/users/5a53639476444b549a7d480d99fc571b/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:16.435598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ca8b8-5d5a-4f2f-90cf-6c42683a05f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84ba8d7532143adbd68abf2fd9c0c42, user_id=5a53639476444b549a7d480d99fc571b, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:16.439919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ca8b8-5d5a-4f2f-90cf-6c42683a05f4 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:16.457687 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1092/2143] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:16 2022] PUT /identity/v3/projects/f84ba8d7532143adbd68abf2fd9c0c42/users/5a53639476444b549a7d480d99fc571b/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:16.473165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-99733c05-4c37-41db-a779-fd26d8aabb01 None None] MFA Rules not processed for user `ee6a9052a75a4d8c9f4a08638685012b`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:16.474699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-99733c05-4c37-41db-a779-fd26d8aabb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:16.494556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeb41c-c77e-40fa-8494-6d355c55bd01 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:16.499844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeb41c-c77e-40fa-8494-6d355c55bd01 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:16.500327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeb41c-c77e-40fa-8494-6d355c55bd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:16.557953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-75eeb41c-c77e-40fa-8494-6d355c55bd01 None None] MFA Rules not processed for user `5a53639476444b549a7d480d99fc571b`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:16.560666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-75eeb41c-c77e-40fa-8494-6d355c55bd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:16.621424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1052/2144] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:16.644433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99733c05-4c37-41db-a779-fd26d8aabb01 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:16.715382 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1093/2145] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:16.747492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99733c05-4c37-41db-a779-fd26d8aabb01 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:16.754646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133f27a-20a5-4251-be49-dc615966a79e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:16.755563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133f27a-20a5-4251-be49-dc615966a79e service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:16.755973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133f27a-20a5-4251-be49-dc615966a79e service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:16.756437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133f27a-20a5-4251-be49-dc615966a79e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:16.771630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133f27a-20a5-4251-be49-dc615966a79e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:16.773883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133f27a-20a5-4251-be49-dc615966a79e service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:16.788114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1053/2146] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:04:16.983292 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeb41c-c77e-40fa-8494-6d355c55bd01 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:16.987791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-75eeb41c-c77e-40fa-8494-6d355c55bd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:17.011124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeb41c-c77e-40fa-8494-6d355c55bd01 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:17.018603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb0b06-7346-4a68-9152-75dedc2b45ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:17.020708 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb0b06-7346-4a68-9152-75dedc2b45ee service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:17.021553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb0b06-7346-4a68-9152-75dedc2b45ee service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:17.022106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb0b06-7346-4a68-9152-75dedc2b45ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:17.040676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb0b06-7346-4a68-9152-75dedc2b45ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:17.044017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb0b06-7346-4a68-9152-75dedc2b45ee service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:17.065587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1094/2147] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:17.850671 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133f27a-20a5-4251-be49-dc615966a79e service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:17.853755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1133f27a-20a5-4251-be49-dc615966a79e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:17.947241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133f27a-20a5-4251-be49-dc615966a79e service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:17.953134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:17.954561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:17.954956 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:17.955340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] PATH_INFO: `/v3/users/70d76a9080964bc2899ab57e10818e3d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:17.956931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d76a9080964bc2899ab57e10818e3d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:17.959214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:18.296978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:18.594950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d76a9080964bc2899ab57e10818e3d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:18.603112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d76a9080964bc2899ab57e10818e3d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:18.603914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d76a9080964bc2899ab57e10818e3d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:18.694269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d76a9080964bc2899ab57e10818e3d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:18.742544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d76a9080964bc2899ab57e10818e3d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:18.767749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1054/2148] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:17 2022] DELETE /identity/v3/users/70d76a9080964bc2899ab57e10818e3d => generated 0 bytes in 918 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:18.892284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb0b06-7346-4a68-9152-75dedc2b45ee service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:18.898618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-20bb0b06-7346-4a68-9152-75dedc2b45ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:18.941946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb0b06-7346-4a68-9152-75dedc2b45ee service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:19.242117 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:19.243710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:19.244287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:19.244798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] PATH_INFO: `/v3/users/9f060b09b83b44939aee161095301d63` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:19.248646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f060b09b83b44939aee161095301d63)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:19.254250 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:19.537798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:19.540240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:19.604284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f060b09b83b44939aee161095301d63'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:19.605645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f060b09b83b44939aee161095301d63'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:19.607775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95e732-9971-4441-9a2c-18ab28f7ab15 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:19.616033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:19.617993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:19.618490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:19.618964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] PATH_INFO: `/v3/projects/0a766cf899434ae98684ffa06f412e2c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:19.620462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f060b09b83b44939aee161095301d63'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:19.621171 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a766cf899434ae98684ffa06f412e2c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:19.621684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f060b09b83b44939aee161095301d63'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:19.624725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:19.757408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-42841b18-be3c-458b-890c-dd4fbdfa8829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f060b09b83b44939aee161095301d63'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:19.798095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1095/2149] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:18 2022] DELETE /identity/v3/users/9f060b09b83b44939aee161095301d63 => generated 0 bytes in 906 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:19.801213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2a3e7-9177-453b-ba38-6ade80fc6d5d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:19.801938 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2a3e7-9177-453b-ba38-6ade80fc6d5d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:19.802529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2a3e7-9177-453b-ba38-6ade80fc6d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:19.825274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a766cf899434ae98684ffa06f412e2c', 'request_id': 'req-586c280a-2e19-4ba4-9e87-55bb6430b11b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:19.837650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a766cf899434ae98684ffa06f412e2c', 'request_id': 'req-586c280a-2e19-4ba4-9e87-55bb6430b11b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:19.847986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a766cf899434ae98684ffa06f412e2c', 'request_id': 'req-586c280a-2e19-4ba4-9e87-55bb6430b11b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:19.850653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d5a2a3e7-9177-453b-ba38-6ade80fc6d5d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:19.853638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d5a2a3e7-9177-453b-ba38-6ade80fc6d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:19.956036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1096/2150] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:19.969831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] The token cache is being invalidate because project 0a766cf899434ae98684ffa06f412e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:19.970617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:19.973395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:19.975768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2a3e7-9177-453b-ba38-6ade80fc6d5d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:19.978322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1055/2151] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:19 2022] DELETE /identity/v3/projects/0a766cf899434ae98684ffa06f412e2c => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:19.979876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d5a2a3e7-9177-453b-ba38-6ade80fc6d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:19.984279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:19.985984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:20.003422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c280a-2e19-4ba4-9e87-55bb6430b11b admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:20.010681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:20.013316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:20.013316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:20.013316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] PATH_INFO: `/v3/users/d4c6f2541f944fbd9c6c5f0bf99e735f` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:20.017263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c6f2541f944fbd9c6c5f0bf99e735f)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:20.019510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:20.106655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2a3e7-9177-453b-ba38-6ade80fc6d5d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:20.111337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48ce60-2a05-435f-861d-51473b2043de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:20.112301 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48ce60-2a05-435f-861d-51473b2043de service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:20.112804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48ce60-2a05-435f-861d-51473b2043de service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:20.113403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48ce60-2a05-435f-861d-51473b2043de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:20.115830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4e48ce60-2a05-435f-861d-51473b2043de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:20.134127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48ce60-2a05-435f-861d-51473b2043de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:20.136061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48ce60-2a05-435f-861d-51473b2043de service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:20.151219 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1097/2152] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:20.204549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c98aa-4f28-4da5-b520-06f38a0b0e49 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:20.204942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c98aa-4f28-4da5-b520-06f38a0b0e49 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:20.205416 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c98aa-4f28-4da5-b520-06f38a0b0e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:20.234596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-456c98aa-4f28-4da5-b520-06f38a0b0e49 None None] MFA Rules not processed for user `ee6a9052a75a4d8c9f4a08638685012b`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:20.242154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-456c98aa-4f28-4da5-b520-06f38a0b0e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:20.287941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:20.391044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1098/2153] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:04:20.463319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c6f2541f944fbd9c6c5f0bf99e735f'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:20.485841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c6f2541f944fbd9c6c5f0bf99e735f'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:20.488819 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c6f2541f944fbd9c6c5f0bf99e735f'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:20.613104 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c6f2541f944fbd9c6c5f0bf99e735f'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:20.664099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c6f2541f944fbd9c6c5f0bf99e735f'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:20.687150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1056/2154] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:19 2022] DELETE /identity/v3/users/d4c6f2541f944fbd9c6c5f0bf99e735f => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:20.843704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c98aa-4f28-4da5-b520-06f38a0b0e49 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:20.848506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-456c98aa-4f28-4da5-b520-06f38a0b0e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:20.885322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c98aa-4f28-4da5-b520-06f38a0b0e49 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:21.055996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:21.062362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:21.062362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:21.062362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] PATH_INFO: `/v3/users/d4a95eee6334412cae58ec4bdad67aad` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:21.063961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a95eee6334412cae58ec4bdad67aad)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:21.075985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:21.488064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a95eee6334412cae58ec4bdad67aad'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:21.488978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a95eee6334412cae58ec4bdad67aad'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:21.514436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a95eee6334412cae58ec4bdad67aad'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:21.517258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a95eee6334412cae58ec4bdad67aad'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:21.637443 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-65f85750-4d18-4304-b72e-2152f4712225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a95eee6334412cae58ec4bdad67aad'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:21.673845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:21.678291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:21.709064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1099/2155] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:20 2022] DELETE /identity/v3/users/d4a95eee6334412cae58ec4bdad67aad => generated 0 bytes in 862 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:21.710359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0711fa-73ea-45cd-a0e9-e304b256d381 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:21.710654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0711fa-73ea-45cd-a0e9-e304b256d381 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:21.713617 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0711fa-73ea-45cd-a0e9-e304b256d381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:21.718552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b99be-e107-44be-b4d0-a630af661fd2 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:21.778087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-ae0711fa-73ea-45cd-a0e9-e304b256d381 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:21.781552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ae0711fa-73ea-45cd-a0e9-e304b256d381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:21.810226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:21.811336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:21.811776 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:21.812280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] PATH_INFO: `/v3/projects/9cba13776ab74f93b1b3ff1550fff39b` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:21.813736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cba13776ab74f93b1b3ff1550fff39b)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:21.815726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:22.001918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1100/2156] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:22.003772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0711fa-73ea-45cd-a0e9-e304b256d381 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:22.007003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ae0711fa-73ea-45cd-a0e9-e304b256d381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:22.038934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cba13776ab74f93b1b3ff1550fff39b', 'request_id': 'req-43a0b74d-ab18-42b5-89bb-bca583c281d1'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:22.040706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0711fa-73ea-45cd-a0e9-e304b256d381 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:22.065777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cba13776ab74f93b1b3ff1550fff39b', 'request_id': 'req-43a0b74d-ab18-42b5-89bb-bca583c281d1'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:22.076760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cba13776ab74f93b1b3ff1550fff39b', 'request_id': 'req-43a0b74d-ab18-42b5-89bb-bca583c281d1'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:22.170442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:22.171457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:22.171953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:22.172546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] PATH_INFO: `/v3/projects/9373f3eee4f844f5bb16ea150d123bb7` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:22.174703 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9373f3eee4f844f5bb16ea150d123bb7)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:22.180565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:22.236383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] The token cache is being invalidate because project 9cba13776ab74f93b1b3ff1550fff39b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:22.240531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:22.241256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:22.249717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1057/2157] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:21 2022] DELETE /identity/v3/projects/9cba13776ab74f93b1b3ff1550fff39b => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:22.249717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:22.249717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:22.374774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0b74d-ab18-42b5-89bb-bca583c281d1 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:22.381866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c185d5-e451-45c3-b301-9f36280f1772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:22.382502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c185d5-e451-45c3-b301-9f36280f1772 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:22.382787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c185d5-e451-45c3-b301-9f36280f1772 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:22.383046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c185d5-e451-45c3-b301-9f36280f1772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:22.385088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-24c185d5-e451-45c3-b301-9f36280f1772 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:22.396957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:22.411182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9373f3eee4f844f5bb16ea150d123bb7', 'request_id': 'req-b3565481-f769-4fef-98e1-2a26b53c0112'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:22.420457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9373f3eee4f844f5bb16ea150d123bb7', 'request_id': 'req-b3565481-f769-4fef-98e1-2a26b53c0112'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:22.421647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c185d5-e451-45c3-b301-9f36280f1772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:22.423917 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c185d5-e451-45c3-b301-9f36280f1772 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:22.451650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1058/2158] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:22.453512 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff348df7-9f8c-4441-ba65-94bf4d2177fc None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:22.453807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff348df7-9f8c-4441-ba65-94bf4d2177fc None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:22.456418 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff348df7-9f8c-4441-ba65-94bf4d2177fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:22.528194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-ff348df7-9f8c-4441-ba65-94bf4d2177fc None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:22.532933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ff348df7-9f8c-4441-ba65-94bf4d2177fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:22.557741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9373f3eee4f844f5bb16ea150d123bb7', 'request_id': 'req-b3565481-f769-4fef-98e1-2a26b53c0112'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:22.603540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] The token cache is being invalidate because project 9373f3eee4f844f5bb16ea150d123bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:22.604342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:22.606995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b3565481-f769-4fef-98e1-2a26b53c0112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:22.622871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1101/2159] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:22 2022] DELETE /identity/v3/projects/9373f3eee4f844f5bb16ea150d123bb7 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:22.626721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5f7e-04de-4ef2-80a6-d10526fbe4ee None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:22.627227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5f7e-04de-4ef2-80a6-d10526fbe4ee None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:22.628069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5f7e-04de-4ef2-80a6-d10526fbe4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:22.699859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-543a5f7e-04de-4ef2-80a6-d10526fbe4ee None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:22.699859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-543a5f7e-04de-4ef2-80a6-d10526fbe4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:22.809123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1059/2160] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:22.814405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17696e73-ae6f-43fa-8eca-dd955f13479e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:22.814677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17696e73-ae6f-43fa-8eca-dd955f13479e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:22.814957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17696e73-ae6f-43fa-8eca-dd955f13479e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:22.868546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-17696e73-ae6f-43fa-8eca-dd955f13479e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:22.869353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-17696e73-ae6f-43fa-8eca-dd955f13479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:22.901928 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1060/2161] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:22.909438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17696e73-ae6f-43fa-8eca-dd955f13479e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:22.925398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1102/2162] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:22.954699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8afb9-d705-4e3d-9c09-aede592e764b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:22.954699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8afb9-d705-4e3d-9c09-aede592e764b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:22.954699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8afb9-d705-4e3d-9c09-aede592e764b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:22.956234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17696e73-ae6f-43fa-8eca-dd955f13479e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:22.966421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf62af8-2a28-4055-801e-78d9947dea6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:22.967474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf62af8-2a28-4055-801e-78d9947dea6d admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:22.967965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf62af8-2a28-4055-801e-78d9947dea6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:22.968497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf62af8-2a28-4055-801e-78d9947dea6d admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:22.972876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf62af8-2a28-4055-801e-78d9947dea6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:22.972876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf62af8-2a28-4055-801e-78d9947dea6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:22.981003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf62af8-2a28-4055-801e-78d9947dea6d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:22.989634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d2c8afb9-d705-4e3d-9c09-aede592e764b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:22.993127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d2c8afb9-d705-4e3d-9c09-aede592e764b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:23.029037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1103/2163] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:04:23.035750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8afb9-d705-4e3d-9c09-aede592e764b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:23.058252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8afb9-d705-4e3d-9c09-aede592e764b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:23.064468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1061/2164] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:23.069528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf62af8-2a28-4055-801e-78d9947dea6d admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:23.078996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c26a6-fc72-460f-b953-d9762f92be77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:23.079296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c26a6-fc72-460f-b953-d9762f92be77 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:23.079476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c26a6-fc72-460f-b953-d9762f92be77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:23.079862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c26a6-fc72-460f-b953-d9762f92be77 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:23.080561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c26a6-fc72-460f-b953-d9762f92be77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:23.080878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c26a6-fc72-460f-b953-d9762f92be77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:23.082662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c26a6-fc72-460f-b953-d9762f92be77 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:23.107877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf62af8-2a28-4055-801e-78d9947dea6d admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:23.121588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1104/2165] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:23.130930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c26a6-fc72-460f-b953-d9762f92be77 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:23.146801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de181f83-2ae8-4baf-ab79-fc03b514361c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:23.147789 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de181f83-2ae8-4baf-ab79-fc03b514361c admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:23.148537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de181f83-2ae8-4baf-ab79-fc03b514361c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:23.149226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de181f83-2ae8-4baf-ab79-fc03b514361c admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:23.150571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de181f83-2ae8-4baf-ab79-fc03b514361c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:23.154264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de181f83-2ae8-4baf-ab79-fc03b514361c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:23.157769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c26a6-fc72-460f-b953-d9762f92be77 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:23.169271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2845820-fbdf-415a-9718-88bef5441230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:23.172209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2845820-fbdf-415a-9718-88bef5441230 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:23.172209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2845820-fbdf-415a-9718-88bef5441230 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:23.172209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2845820-fbdf-415a-9718-88bef5441230 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:23.174110 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2845820-fbdf-415a-9718-88bef5441230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:23.179358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2845820-fbdf-415a-9718-88bef5441230 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:23.368136 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1105/2166] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:23 2022] POST /identity/v3/projects => generated 397 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:04:23.374409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2845820-fbdf-415a-9718-88bef5441230 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:23.374674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1062/2167] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:23 2022] POST /identity/v3/projects => generated 369 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:23.375666 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de181f83-2ae8-4baf-ab79-fc03b514361c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:23.480216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2845820-fbdf-415a-9718-88bef5441230 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:23.486998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a727-f1b9-48cb-9a7b-b8d0898a7204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:23.487648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a727-f1b9-48cb-9a7b-b8d0898a7204 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:23.487943 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a727-f1b9-48cb-9a7b-b8d0898a7204 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:23.488241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a727-f1b9-48cb-9a7b-b8d0898a7204 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:23.489355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718a727-f1b9-48cb-9a7b-b8d0898a7204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:23.491735 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718a727-f1b9-48cb-9a7b-b8d0898a7204 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:23.520586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de181f83-2ae8-4baf-ab79-fc03b514361c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:23.525611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2bdd5-7bd5-465b-b698-2610b66de0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:23.526286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2bdd5-7bd5-465b-b698-2610b66de0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:23.526583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2bdd5-7bd5-465b-b698-2610b66de0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:23.527023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2bdd5-7bd5-465b-b698-2610b66de0e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:23.527681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2bdd5-7bd5-465b-b698-2610b66de0e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:23.529685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2bdd5-7bd5-465b-b698-2610b66de0e5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:23.675528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1106/2168] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:23 2022] POST /identity/v3/users => generated 299 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:23.677754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a727-f1b9-48cb-9a7b-b8d0898a7204 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:23.681327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1063/2169] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:23 2022] POST /identity/v3/users => generated 285 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:23.683685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2bdd5-7bd5-465b-b698-2610b66de0e5 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:23.688238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-15b2bdd5-7bd5-465b-b698-2610b66de0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:23.708251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a727-f1b9-48cb-9a7b-b8d0898a7204 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:23.715915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2bdd5-7bd5-465b-b698-2610b66de0e5 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:23.724156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542213f5-0974-48e0-b471-ff9f9f225cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:23.724840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542213f5-0974-48e0-b471-ff9f9f225cbc admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:23.725128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542213f5-0974-48e0-b471-ff9f9f225cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:23.725407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542213f5-0974-48e0-b471-ff9f9f225cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:23.726174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542213f5-0974-48e0-b471-ff9f9f225cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:23.726769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:23.727897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542213f5-0974-48e0-b471-ff9f9f225cbc admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:23.728294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:23.729046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:23.730413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] PATH_INFO: `/v3/projects/7bae5c18c4e54529b2c641f1197a4717` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:23.735664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bae5c18c4e54529b2c641f1197a4717)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:23.739779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:23.756976 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1107/2170] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:23.756976 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542213f5-0974-48e0-b471-ff9f9f225cbc admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:23.809504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542213f5-0974-48e0-b471-ff9f9f225cbc admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:23.818963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9928cb3-c4a8-4b54-862f-0f35524d2971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:23.819603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9928cb3-c4a8-4b54-862f-0f35524d2971 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:23.819879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9928cb3-c4a8-4b54-862f-0f35524d2971 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:23.820164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9928cb3-c4a8-4b54-862f-0f35524d2971 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:23.824868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9928cb3-c4a8-4b54-862f-0f35524d2971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:23.827930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9928cb3-c4a8-4b54-862f-0f35524d2971 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:23.871641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1108/2171] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:23.875894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9928cb3-c4a8-4b54-862f-0f35524d2971 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:23.898709 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9928cb3-c4a8-4b54-862f-0f35524d2971 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:23.916866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bae5c18c4e54529b2c641f1197a4717', 'request_id': 'req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:23.946632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bae5c18c4e54529b2c641f1197a4717', 'request_id': 'req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:23.967416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bae5c18c4e54529b2c641f1197a4717', 'request_id': 'req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:24.053156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f4c52-9e78-4066-9922-5001d4d4a8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:24.053850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f4c52-9e78-4066-9922-5001d4d4a8fb admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:24.054040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f4c52-9e78-4066-9922-5001d4d4a8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:24.054477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f4c52-9e78-4066-9922-5001d4d4a8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:24.055313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f4c52-9e78-4066-9922-5001d4d4a8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:24.057182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f4c52-9e78-4066-9922-5001d4d4a8fb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:24.081712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1109/2172] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:24.083019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f4c52-9e78-4066-9922-5001d4d4a8fb admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:24.083818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] The token cache is being invalidate because project 7bae5c18c4e54529b2c641f1197a4717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:24.086051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:24.086051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:24.096385 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1064/2173] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:23 2022] DELETE /identity/v3/projects/7bae5c18c4e54529b2c641f1197a4717 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:24.097886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:24.102441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:24.107629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f4c52-9e78-4066-9922-5001d4d4a8fb admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:24.135566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab333df-9dd2-400e-9cf4-7a4a17210b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:24.138935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab333df-9dd2-400e-9cf4-7a4a17210b35 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:24.139371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab333df-9dd2-400e-9cf4-7a4a17210b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:24.139755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab333df-9dd2-400e-9cf4-7a4a17210b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:24.140595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab333df-9dd2-400e-9cf4-7a4a17210b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:24.145476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab333df-9dd2-400e-9cf4-7a4a17210b35 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:24.158520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b90da5-49cf-4e35-8cc5-7398c6d9c6fe admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:24.181523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1110/2174] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:24.181523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab333df-9dd2-400e-9cf4-7a4a17210b35 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:24.192476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7419fd3-bb89-4c06-8121-41dff1101a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:24.193785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7419fd3-bb89-4c06-8121-41dff1101a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:24.194395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7419fd3-bb89-4c06-8121-41dff1101a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:24.194583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4ab333df-9dd2-400e-9cf4-7a4a17210b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:24.195206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7419fd3-bb89-4c06-8121-41dff1101a27 admin admin] PATH_INFO: `/v3/projects/ec632aa04f4b43b4b0afdade22257a3d/users/7adbe54293fc49a985566e04258e5580/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:24.232519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7419fd3-bb89-4c06-8121-41dff1101a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec632aa04f4b43b4b0afdade22257a3d, user_id=7adbe54293fc49a985566e04258e5580, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:24.234334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab333df-9dd2-400e-9cf4-7a4a17210b35 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:24.238075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7419fd3-bb89-4c06-8121-41dff1101a27 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:24.248917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:24.249972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:24.251042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:24.251499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] PATH_INFO: `/v3/users/5a53639476444b549a7d480d99fc571b` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:24.253447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a53639476444b549a7d480d99fc571b)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:24.255928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:24.360474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1065/2175] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:24 2022] PUT /identity/v3/projects/ec632aa04f4b43b4b0afdade22257a3d/users/7adbe54293fc49a985566e04258e5580/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:24.362052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7419fd3-bb89-4c06-8121-41dff1101a27 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:24.373296 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e7419fd3-bb89-4c06-8121-41dff1101a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:24.483973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7419fd3-bb89-4c06-8121-41dff1101a27 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:24.489241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010712e-88e7-440d-8ff3-23bb35a42b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:24.489920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010712e-88e7-440d-8ff3-23bb35a42b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:24.490209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010712e-88e7-440d-8ff3-23bb35a42b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:24.490477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010712e-88e7-440d-8ff3-23bb35a42b1e admin admin] PATH_INFO: `/v3/projects/7317f7960dc7424fac44883851792b08/users/69e539a5ddf04791b78a5d2e68899e7b/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:24.518205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010712e-88e7-440d-8ff3-23bb35a42b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7317f7960dc7424fac44883851792b08, user_id=69e539a5ddf04791b78a5d2e68899e7b, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:24.523871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010712e-88e7-440d-8ff3-23bb35a42b1e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:24.609666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a53639476444b549a7d480d99fc571b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:24.610764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a53639476444b549a7d480d99fc571b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:24.615085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:24.622676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a53639476444b549a7d480d99fc571b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:24.623725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a53639476444b549a7d480d99fc571b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:24.711893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3e142d93-7410-4e86-b71c-0b2e0b530e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a53639476444b549a7d480d99fc571b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:24.752539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1066/2176] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:24 2022] PUT /identity/v3/projects/7317f7960dc7424fac44883851792b08/users/69e539a5ddf04791b78a5d2e68899e7b/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:24.760429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846646b2-5415-4048-ac6f-3563d495a532 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:24.761351 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846646b2-5415-4048-ac6f-3563d495a532 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:24.761857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846646b2-5415-4048-ac6f-3563d495a532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:24.777773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1111/2177] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:24 2022] DELETE /identity/v3/users/5a53639476444b549a7d480d99fc571b => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:24.781120 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f36f-37f4-4941-9223-a21ab97dcb07 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:24.782202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f36f-37f4-4941-9223-a21ab97dcb07 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:24.783221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f36f-37f4-4941-9223-a21ab97dcb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:24.878331 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-846646b2-5415-4048-ac6f-3563d495a532 None None] MFA Rules not processed for user `7adbe54293fc49a985566e04258e5580`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:24.884829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-846646b2-5415-4048-ac6f-3563d495a532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:24.949631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4796f36f-37f4-4941-9223-a21ab97dcb07 None None] MFA Rules not processed for user `69e539a5ddf04791b78a5d2e68899e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:24.952991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4796f36f-37f4-4941-9223-a21ab97dcb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:25.174287 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1067/2178] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:25.224575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846646b2-5415-4048-ac6f-3563d495a532 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:25.226657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-846646b2-5415-4048-ac6f-3563d495a532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:25.293746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1112/2179] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:25.319109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796f36f-37f4-4941-9223-a21ab97dcb07 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:25.321895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4796f36f-37f4-4941-9223-a21ab97dcb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:25.406757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846646b2-5415-4048-ac6f-3563d495a532 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:25.432033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e5ee8-48bc-47a5-a332-8712501af97d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:25.436206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e5ee8-48bc-47a5-a332-8712501af97d service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:25.439517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e5ee8-48bc-47a5-a332-8712501af97d service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:25.439517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e5ee8-48bc-47a5-a332-8712501af97d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:25.482034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e5ee8-48bc-47a5-a332-8712501af97d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:25.492490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e5ee8-48bc-47a5-a332-8712501af97d service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:25.520696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796f36f-37f4-4941-9223-a21ab97dcb07 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:25.526279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d15ab-d913-4b7c-a59d-db6dbbfd3530 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454fb811f6e34e1d881d4948ede058b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:25.527382 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d15ab-d913-4b7c-a59d-db6dbbfd3530 service glance] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:25.527789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d15ab-d913-4b7c-a59d-db6dbbfd3530 service glance] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:25.528493 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d15ab-d913-4b7c-a59d-db6dbbfd3530 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:25.529139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1068/2180] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:25 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:25.575001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d15ab-d913-4b7c-a59d-db6dbbfd3530 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:25.580696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d15ab-d913-4b7c-a59d-db6dbbfd3530 service glance] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:25.632338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1113/2181] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:04:25.998928 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e5ee8-48bc-47a5-a332-8712501af97d service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:26.004599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1b9e5ee8-48bc-47a5-a332-8712501af97d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:26.223426 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfa954-0f2d-4ff9-8aa6-b54110d0c305 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:26.224867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfa954-0f2d-4ff9-8aa6-b54110d0c305 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:26.225643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfa954-0f2d-4ff9-8aa6-b54110d0c305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:26.240694 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e5ee8-48bc-47a5-a332-8712501af97d service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:26.251895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:26.254107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:26.254813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:26.255257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] PATH_INFO: `/v3/projects/f84ba8d7532143adbd68abf2fd9c0c42` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:26.260068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84ba8d7532143adbd68abf2fd9c0c42)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:26.262575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:26.297378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-24cfa954-0f2d-4ff9-8aa6-b54110d0c305 None None] MFA Rules not processed for user `f3f607e48e534dd09c3619982978e9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:26.310966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-24cfa954-0f2d-4ff9-8aa6-b54110d0c305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:26.455504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1114/2182] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:26.476081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bf8a6-9bb7-4646-9edd-4b8805c76c46 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:26.476847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bf8a6-9bb7-4646-9edd-4b8805c76c46 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:26.477136 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bf8a6-9bb7-4646-9edd-4b8805c76c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:26.483756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84ba8d7532143adbd68abf2fd9c0c42', 'request_id': 'req-16b20979-8517-420b-af05-c759254ee501'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:26.503943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84ba8d7532143adbd68abf2fd9c0c42', 'request_id': 'req-16b20979-8517-420b-af05-c759254ee501'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:26.506753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-894bf8a6-9bb7-4646-9edd-4b8805c76c46 None None] MFA Rules not processed for user `f3f607e48e534dd09c3619982978e9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:26.508296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-894bf8a6-9bb7-4646-9edd-4b8805c76c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:26.525206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84ba8d7532143adbd68abf2fd9c0c42', 'request_id': 'req-16b20979-8517-420b-af05-c759254ee501'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:26.594007 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1115/2183] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:26.606081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bf8a6-9bb7-4646-9edd-4b8805c76c46 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:26.607669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-894bf8a6-9bb7-4646-9edd-4b8805c76c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:26.636519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] The token cache is being invalidate because project f84ba8d7532143adbd68abf2fd9c0c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:26.638434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:26.639141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-16b20979-8517-420b-af05-c759254ee501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:26.642444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1069/2184] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:25 2022] DELETE /identity/v3/projects/f84ba8d7532143adbd68abf2fd9c0c42 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:26.648764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49f705-cc69-43e2-8f4b-3ff4f1170a08 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:26.650479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49f705-cc69-43e2-8f4b-3ff4f1170a08 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:26.651302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49f705-cc69-43e2-8f4b-3ff4f1170a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:26.684080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-4b49f705-cc69-43e2-8f4b-3ff4f1170a08 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:26.685800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4b49f705-cc69-43e2-8f4b-3ff4f1170a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:26.745091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bf8a6-9bb7-4646-9edd-4b8805c76c46 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:26.755441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a8191-ed43-4528-8ad0-4cfe59bd2e61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:26.759127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a8191-ed43-4528-8ad0-4cfe59bd2e61 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:26.759127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a8191-ed43-4528-8ad0-4cfe59bd2e61 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:26.759127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a8191-ed43-4528-8ad0-4cfe59bd2e61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:26.762061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ee0a8191-ed43-4528-8ad0-4cfe59bd2e61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:26.783135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a8191-ed43-4528-8ad0-4cfe59bd2e61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:26.786191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a8191-ed43-4528-8ad0-4cfe59bd2e61 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:26.806055 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1116/2185] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:26.831630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:26.838892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1070/2186] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:26.848418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e1506-6885-4b22-ae89-24a715b23d1e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:26.848418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e1506-6885-4b22-ae89-24a715b23d1e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:26.848418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e1506-6885-4b22-ae89-24a715b23d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:26.871940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-682e1506-6885-4b22-ae89-24a715b23d1e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:26.877028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-682e1506-6885-4b22-ae89-24a715b23d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:26.900397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1117/2187] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:04:26.901673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49f705-cc69-43e2-8f4b-3ff4f1170a08 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:26.922616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49f705-cc69-43e2-8f4b-3ff4f1170a08 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:26.966598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718aeb-a97a-4c3e-bd3d-0b8f028b25e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:26.967378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718aeb-a97a-4c3e-bd3d-0b8f028b25e1 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:26.967731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718aeb-a97a-4c3e-bd3d-0b8f028b25e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:26.968076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718aeb-a97a-4c3e-bd3d-0b8f028b25e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:26.968951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a718aeb-a97a-4c3e-bd3d-0b8f028b25e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:26.969328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a718aeb-a97a-4c3e-bd3d-0b8f028b25e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:26.971198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a718aeb-a97a-4c3e-bd3d-0b8f028b25e1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:26.988315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1071/2188] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:26.995111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e1506-6885-4b22-ae89-24a715b23d1e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:27.012628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e1506-6885-4b22-ae89-24a715b23d1e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:27.021796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f054f-474d-479a-93d4-6f36f0be211f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:27.022918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f054f-474d-479a-93d4-6f36f0be211f admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:27.023428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f054f-474d-479a-93d4-6f36f0be211f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:27.024039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f054f-474d-479a-93d4-6f36f0be211f admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:27.025373 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f054f-474d-479a-93d4-6f36f0be211f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:27.029542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f054f-474d-479a-93d4-6f36f0be211f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:27.434273 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1118/2189] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:26 2022] POST /identity/v3/projects => generated 373 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:27.440197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718aeb-a97a-4c3e-bd3d-0b8f028b25e1 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:27.463264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718aeb-a97a-4c3e-bd3d-0b8f028b25e1 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:27.596440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643464e0-f0ba-4e75-b84b-b044e2485710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:27.597501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643464e0-f0ba-4e75-b84b-b044e2485710 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:27.597922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643464e0-f0ba-4e75-b84b-b044e2485710 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:27.599124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643464e0-f0ba-4e75-b84b-b044e2485710 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:27.600301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643464e0-f0ba-4e75-b84b-b044e2485710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:27.603607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643464e0-f0ba-4e75-b84b-b044e2485710 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:27.831613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1072/2190] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:27 2022] POST /identity/v3/users => generated 287 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:27.837596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f054f-474d-479a-93d4-6f36f0be211f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:27.850625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f054f-474d-479a-93d4-6f36f0be211f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:27.856872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d594-b930-4982-ae3d-395e4fffad0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:27.857572 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d594-b930-4982-ae3d-395e4fffad0f admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:27.859019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d594-b930-4982-ae3d-395e4fffad0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:27.859873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d594-b930-4982-ae3d-395e4fffad0f admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:27.860603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8d594-b930-4982-ae3d-395e4fffad0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:27.862325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8d594-b930-4982-ae3d-395e4fffad0f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:27.882710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1119/2191] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:27.889155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643464e0-f0ba-4e75-b84b-b044e2485710 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:27.908829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643464e0-f0ba-4e75-b84b-b044e2485710 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:27.914699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26840bf0-f4f5-486d-9d05-dd561c662e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:27.917730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26840bf0-f4f5-486d-9d05-dd561c662e81 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:27.917730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26840bf0-f4f5-486d-9d05-dd561c662e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:27.917730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26840bf0-f4f5-486d-9d05-dd561c662e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:27.917730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26840bf0-f4f5-486d-9d05-dd561c662e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:27.918595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26840bf0-f4f5-486d-9d05-dd561c662e81 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:27.933498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1073/2192] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:27.937878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d594-b930-4982-ae3d-395e4fffad0f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:27.949488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d594-b930-4982-ae3d-395e4fffad0f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:27.955998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd411bb8-45de-43b6-bef2-979258167600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:27.963297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd411bb8-45de-43b6-bef2-979258167600 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:27.963297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd411bb8-45de-43b6-bef2-979258167600 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:27.963297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd411bb8-45de-43b6-bef2-979258167600 admin admin] PATH_INFO: `/v3/projects/aceec22ab6924d43a326971b68104492/users/a46fb8a2d5764e9fa5d9138918dc5a19/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:27.989469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd411bb8-45de-43b6-bef2-979258167600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aceec22ab6924d43a326971b68104492, user_id=a46fb8a2d5764e9fa5d9138918dc5a19, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:28.006361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd411bb8-45de-43b6-bef2-979258167600 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:28.110875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b00c8-2b1a-4285-8a90-429d6422524a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:28.111340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b00c8-2b1a-4285-8a90-429d6422524a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:28.111804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b00c8-2b1a-4285-8a90-429d6422524a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:28.151405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1120/2193] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:27 2022] PUT /identity/v3/projects/aceec22ab6924d43a326971b68104492/users/a46fb8a2d5764e9fa5d9138918dc5a19/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:28.166691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7a3b00c8-2b1a-4285-8a90-429d6422524a None None] MFA Rules not processed for user `f3f607e48e534dd09c3619982978e9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:28.169340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7a3b00c8-2b1a-4285-8a90-429d6422524a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:28.181364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6632-24b3-49f6-b55a-b270a8df9d48 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:28.181364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6632-24b3-49f6-b55a-b270a8df9d48 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:28.181364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6632-24b3-49f6-b55a-b270a8df9d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:28.287953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-7e3b6632-24b3-49f6-b55a-b270a8df9d48 None None] MFA Rules not processed for user `a46fb8a2d5764e9fa5d9138918dc5a19`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:28.294260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7e3b6632-24b3-49f6-b55a-b270a8df9d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:28.380561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1074/2194] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:04:28.475505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1121/2195] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:28.482238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b00c8-2b1a-4285-8a90-429d6422524a None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:28.712045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b00c8-2b1a-4285-8a90-429d6422524a None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:28.720626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edc8f2-6595-4d71-afef-adc445715225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:28.721816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edc8f2-6595-4d71-afef-adc445715225 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:28.722402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edc8f2-6595-4d71-afef-adc445715225 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:28.723025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edc8f2-6595-4d71-afef-adc445715225 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:28.724308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edc8f2-6595-4d71-afef-adc445715225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:28.729053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edc8f2-6595-4d71-afef-adc445715225 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:28.969545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1075/2196] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:28 2022] POST /identity/v3/projects => generated 375 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:28.975799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6632-24b3-49f6-b55a-b270a8df9d48 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:28.996779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6632-24b3-49f6-b55a-b270a8df9d48 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:29.158364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637c3c6-b8b0-45cc-840a-82c417612faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:29.159023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637c3c6-b8b0-45cc-840a-82c417612faf admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:29.159310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637c3c6-b8b0-45cc-840a-82c417612faf admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:29.159777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637c3c6-b8b0-45cc-840a-82c417612faf admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:29.160497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637c3c6-b8b0-45cc-840a-82c417612faf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:29.162560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637c3c6-b8b0-45cc-840a-82c417612faf admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:29.404529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1122/2197] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:28 2022] POST /identity/v3/users => generated 294 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:04:29.408528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edc8f2-6595-4d71-afef-adc445715225 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:29.419975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edc8f2-6595-4d71-afef-adc445715225 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:29.425534 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c55ec-4356-421a-be89-41e827015318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:29.426249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c55ec-4356-421a-be89-41e827015318 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:29.434770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c55ec-4356-421a-be89-41e827015318 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:29.434770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c55ec-4356-421a-be89-41e827015318 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:29.434770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c55ec-4356-421a-be89-41e827015318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:29.434770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c55ec-4356-421a-be89-41e827015318 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:29.444636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1076/2198] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:04:29.449624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637c3c6-b8b0-45cc-840a-82c417612faf admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:29.461034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637c3c6-b8b0-45cc-840a-82c417612faf admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:29.466503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6389d5-050b-4633-aa1f-24868722b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:29.467338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6389d5-050b-4633-aa1f-24868722b7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:29.467711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6389d5-050b-4633-aa1f-24868722b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:29.468095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6389d5-050b-4633-aa1f-24868722b7d7 admin admin] PATH_INFO: `/v3/projects/3b79d576377941c6b9b1cbaac66d50ac/users/9f2c089c99aa42878a4ba624be87b58c/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:29.498483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6389d5-050b-4633-aa1f-24868722b7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b79d576377941c6b9b1cbaac66d50ac, user_id=9f2c089c99aa42878a4ba624be87b58c, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:29.502801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6389d5-050b-4633-aa1f-24868722b7d7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:29.716545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1123/2199] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:29 2022] PUT /identity/v3/projects/3b79d576377941c6b9b1cbaac66d50ac/users/9f2c089c99aa42878a4ba624be87b58c/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:29.726651 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c55ec-4356-421a-be89-41e827015318 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:29.770197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c55ec-4356-421a-be89-41e827015318 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:29.801942 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:29.979779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd721f1-a4d6-4426-999c-e279160fd139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:29.981877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd721f1-a4d6-4426-999c-e279160fd139 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:29.982292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd721f1-a4d6-4426-999c-e279160fd139 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:29.982650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd721f1-a4d6-4426-999c-e279160fd139 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:29.983797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd721f1-a4d6-4426-999c-e279160fd139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:29.986135 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd721f1-a4d6-4426-999c-e279160fd139 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:30.008464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1077/2200] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:04:30.010265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6389d5-050b-4633-aa1f-24868722b7d7 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:30.028838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6389d5-050b-4633-aa1f-24868722b7d7 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:30.040809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20800240-c98a-4413-a204-98370a9e2d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:30.041499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20800240-c98a-4413-a204-98370a9e2d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:30.041786 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20800240-c98a-4413-a204-98370a9e2d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:30.042070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20800240-c98a-4413-a204-98370a9e2d18 admin admin] PATH_INFO: `/v3/projects/3b79d576377941c6b9b1cbaac66d50ac/users/9f2c089c99aa42878a4ba624be87b58c/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:30.046059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20800240-c98a-4413-a204-98370a9e2d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b79d576377941c6b9b1cbaac66d50ac, user_id=9f2c089c99aa42878a4ba624be87b58c, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:30.048881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20800240-c98a-4413-a204-98370a9e2d18 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:30.070083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1124/2201] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:30 2022] PUT /identity/v3/projects/3b79d576377941c6b9b1cbaac66d50ac/users/9f2c089c99aa42878a4ba624be87b58c/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:30.090466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8c1d0-b45c-4849-8a9c-5b2097d73ec6 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:30.091251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8c1d0-b45c-4849-8a9c-5b2097d73ec6 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:30.092239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8c1d0-b45c-4849-8a9c-5b2097d73ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:30.154313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-cbc8c1d0-b45c-4849-8a9c-5b2097d73ec6 None None] MFA Rules not processed for user `9f2c089c99aa42878a4ba624be87b58c`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:30.157236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cbc8c1d0-b45c-4849-8a9c-5b2097d73ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:30.415081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1078/2202] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:30.430432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20800240-c98a-4413-a204-98370a9e2d18 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:30.432798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-20800240-c98a-4413-a204-98370a9e2d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:30.704890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20800240-c98a-4413-a204-98370a9e2d18 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:30.716637 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4426a6-490c-4676-bfb4-c27fa37e7ad5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:30.717569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4426a6-490c-4676-bfb4-c27fa37e7ad5 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:30.717992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4426a6-490c-4676-bfb4-c27fa37e7ad5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:30.719879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4426a6-490c-4676-bfb4-c27fa37e7ad5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:30.917862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4426a6-490c-4676-bfb4-c27fa37e7ad5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:30.921079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4426a6-490c-4676-bfb4-c27fa37e7ad5 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:30.943748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1125/2203] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:32.104108 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:33.509172 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:33.933211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:34.111661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:34.369390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:34.430547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:34.563220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:34.625026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:34.840978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:35.089543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:35.460701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:35.752937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:36.784816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:36.848536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:37.156782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:37.184704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:37.334965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:37.375358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:37.436300 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:37.470737 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:37.556040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:37.641987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:42.356000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8c1d0-b45c-4849-8a9c-5b2097d73ec6 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:42.476677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8c1d0-b45c-4849-8a9c-5b2097d73ec6 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:42.487014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:42.488564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:42.489064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:42.489546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] PATH_INFO: `/v3/users/a46fb8a2d5764e9fa5d9138918dc5a19` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:42.493448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a46fb8a2d5764e9fa5d9138918dc5a19)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:42.497134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:42.963407 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a46fb8a2d5764e9fa5d9138918dc5a19'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:42.990479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a46fb8a2d5764e9fa5d9138918dc5a19'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:42.990479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a46fb8a2d5764e9fa5d9138918dc5a19'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:43.117813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a46fb8a2d5764e9fa5d9138918dc5a19'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:43.157973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a46fb8a2d5764e9fa5d9138918dc5a19'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:43.171540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1079/2204] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:42 2022] DELETE /identity/v3/users/a46fb8a2d5764e9fa5d9138918dc5a19 => generated 0 bytes in 818 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:43.181846 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803f4f8-fa96-448f-9167-058c92b35d9d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:43.182705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803f4f8-fa96-448f-9167-058c92b35d9d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:43.183004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803f4f8-fa96-448f-9167-058c92b35d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:43.223318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1803f4f8-fa96-448f-9167-058c92b35d9d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:43.226498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1803f4f8-fa96-448f-9167-058c92b35d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:43.333804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1126/2205] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:04:43.341507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:43.343608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:43.436239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277434b-a886-4054-ab47-bcfc6404f0a8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:43.442425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019ba75-b18b-4897-93fb-30dc343e204d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:43.443468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019ba75-b18b-4897-93fb-30dc343e204d service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:43.443834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019ba75-b18b-4897-93fb-30dc343e204d service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:43.444196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019ba75-b18b-4897-93fb-30dc343e204d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:43.461306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019ba75-b18b-4897-93fb-30dc343e204d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:43.464627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019ba75-b18b-4897-93fb-30dc343e204d service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:43.486797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1080/2206] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:04:43.830174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803f4f8-fa96-448f-9167-058c92b35d9d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:43.831621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1803f4f8-fa96-448f-9167-058c92b35d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:43.847230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803f4f8-fa96-448f-9167-058c92b35d9d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:43.852923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:43.853745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:43.854211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:43.854546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] PATH_INFO: `/v3/users/7adbe54293fc49a985566e04258e5580` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:43.855897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7adbe54293fc49a985566e04258e5580)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:43.858436 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:44.250828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7adbe54293fc49a985566e04258e5580'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:44.252103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7adbe54293fc49a985566e04258e5580'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:44.273117 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7adbe54293fc49a985566e04258e5580'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:44.278422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7adbe54293fc49a985566e04258e5580'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:44.384504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7adbe54293fc49a985566e04258e5580'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:44.420538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1127/2207] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:43 2022] DELETE /identity/v3/users/7adbe54293fc49a985566e04258e5580 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:44.425320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529d0b9-3baf-454b-9b29-07feeca1ad6e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:44.425851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529d0b9-3baf-454b-9b29-07feeca1ad6e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:44.426314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529d0b9-3baf-454b-9b29-07feeca1ad6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:44.437688 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:44.451890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-5529d0b9-3baf-454b-9b29-07feeca1ad6e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:44.454810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5529d0b9-3baf-454b-9b29-07feeca1ad6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:44.538126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1081/2208] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:44.554605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:44.557884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:44.601044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529d0b9-3baf-454b-9b29-07feeca1ad6e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:44.604141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5529d0b9-3baf-454b-9b29-07feeca1ad6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:44.630552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529d0b9-3baf-454b-9b29-07feeca1ad6e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:44.642000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:44.642935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:44.643219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:44.643654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] PATH_INFO: `/v3/projects/aceec22ab6924d43a326971b68104492` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:44.645158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aceec22ab6924d43a326971b68104492)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:44.647383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:44.685782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1d9a6-1d29-4617-a1a8-56cd1774d16f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:44.695420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47dc32-7885-4e1f-b5d0-3ac8812a8e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:44.696736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47dc32-7885-4e1f-b5d0-3ac8812a8e4c service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:44.698141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47dc32-7885-4e1f-b5d0-3ac8812a8e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:44.700634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47dc32-7885-4e1f-b5d0-3ac8812a8e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:44.725620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47dc32-7885-4e1f-b5d0-3ac8812a8e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:44.729089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47dc32-7885-4e1f-b5d0-3ac8812a8e4c service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:44.751318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1128/2209] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:44.937474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aceec22ab6924d43a326971b68104492', 'request_id': 'req-40a61049-56b2-4c28-bcb7-47416fdd791a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:44.951095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aceec22ab6924d43a326971b68104492', 'request_id': 'req-40a61049-56b2-4c28-bcb7-47416fdd791a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:44.960844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aceec22ab6924d43a326971b68104492', 'request_id': 'req-40a61049-56b2-4c28-bcb7-47416fdd791a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.137146 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] The token cache is being invalidate because project aceec22ab6924d43a326971b68104492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:45.137602 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.140651 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.143780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1082/2210] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:44 2022] DELETE /identity/v3/projects/aceec22ab6924d43a326971b68104492 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:45.149835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47dc32-7885-4e1f-b5d0-3ac8812a8e4c service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:45.158231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:45.175739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47dc32-7885-4e1f-b5d0-3ac8812a8e4c service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:45.183796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:45.190146 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:45.387071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a61049-56b2-4c28-bcb7-47416fdd791a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:45.402521 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:45.403600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:45.404115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:45.404646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] PATH_INFO: `/v3/users/f50449b73a164ad6b7fb2b01c82e7255` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:45.417071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50449b73a164ad6b7fb2b01c82e7255)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:45.422640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:45.444946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:45.446104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:45.450244 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:45.452047 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] PATH_INFO: `/v3/users/9f2c089c99aa42878a4ba624be87b58c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:45.464634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f2c089c99aa42878a4ba624be87b58c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:45.478116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:45.746946 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50449b73a164ad6b7fb2b01c82e7255'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.767729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50449b73a164ad6b7fb2b01c82e7255'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.769434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50449b73a164ad6b7fb2b01c82e7255'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.875882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50449b73a164ad6b7fb2b01c82e7255'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.901848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f2c089c99aa42878a4ba624be87b58c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.905033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f2c089c99aa42878a4ba624be87b58c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.905539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50449b73a164ad6b7fb2b01c82e7255'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.920541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1083/2211] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:45 2022] DELETE /identity/v3/users/f50449b73a164ad6b7fb2b01c82e7255 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:45.922840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:45.924763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:45.937845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f2c089c99aa42878a4ba624be87b58c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:45.945104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f2c089c99aa42878a4ba624be87b58c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:46.042184 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4efc2596-9bed-433b-b302-7104d9a32301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f2c089c99aa42878a4ba624be87b58c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:46.150316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1129/2212] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:04:45 2022] DELETE /identity/v3/users/9f2c089c99aa42878a4ba624be87b58c => generated 0 bytes in 1002 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:46.153742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac9a0-2b93-40ca-a432-754208cbda03 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:46.154431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac9a0-2b93-40ca-a432-754208cbda03 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:46.164550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac9a0-2b93-40ca-a432-754208cbda03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:46.234708 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4c1ac9a0-2b93-40ca-a432-754208cbda03 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:46.236065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4c1ac9a0-2b93-40ca-a432-754208cbda03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:46.281091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a7fa-def1-494c-a060-d9b852749c93 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:46.340143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1130/2213] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:46.389641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ac9a0-2b93-40ca-a432-754208cbda03 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:46.391862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4c1ac9a0-2b93-40ca-a432-754208cbda03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:46.435091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:46.435911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:46.436378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:46.436767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] PATH_INFO: `/v3/projects/ec632aa04f4b43b4b0afdade22257a3d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:46.439371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec632aa04f4b43b4b0afdade22257a3d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:46.445833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:46.501403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:46.526511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ac9a0-2b93-40ca-a432-754208cbda03 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:46.533752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b630b-5d21-42c8-b53a-b1725eb8e0bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:46.534939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b630b-5d21-42c8-b53a-b1725eb8e0bc service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:46.535504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b630b-5d21-42c8-b53a-b1725eb8e0bc service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:46.536071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b630b-5d21-42c8-b53a-b1725eb8e0bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:46.553169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b630b-5d21-42c8-b53a-b1725eb8e0bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:46.555920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b630b-5d21-42c8-b53a-b1725eb8e0bc service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:46.575706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1131/2214] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:04:46.719855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec632aa04f4b43b4b0afdade22257a3d', 'request_id': 'req-f1e58a92-4366-4193-bbcb-9de897486b6a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:46.740162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec632aa04f4b43b4b0afdade22257a3d', 'request_id': 'req-f1e58a92-4366-4193-bbcb-9de897486b6a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:46.753283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec632aa04f4b43b4b0afdade22257a3d', 'request_id': 'req-f1e58a92-4366-4193-bbcb-9de897486b6a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:46.906860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] The token cache is being invalidate because project ec632aa04f4b43b4b0afdade22257a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:46.906860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:46.908382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:46.912971 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1084/2215] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:45 2022] DELETE /identity/v3/projects/ec632aa04f4b43b4b0afdade22257a3d => generated 0 bytes in 991 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:46.918768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb76e9-9ad0-4289-8431-f17598a8574f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:46.919086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb76e9-9ad0-4289-8431-f17598a8574f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:46.919428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb76e9-9ad0-4289-8431-f17598a8574f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:46.959879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-95bb76e9-9ad0-4289-8431-f17598a8574f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:46.959879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-95bb76e9-9ad0-4289-8431-f17598a8574f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:47.023801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:47.025674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:47.142422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1132/2216] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:04:47.145625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9caa1c-98c5-49a3-a55d-6d20dfdc6eea None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:47.146931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9caa1c-98c5-49a3-a55d-6d20dfdc6eea None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:47.147327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9caa1c-98c5-49a3-a55d-6d20dfdc6eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:47.195800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9f9caa1c-98c5-49a3-a55d-6d20dfdc6eea None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:47.201602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9f9caa1c-98c5-49a3-a55d-6d20dfdc6eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:47.226250 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1133/2217] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:04:47.227434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9caa1c-98c5-49a3-a55d-6d20dfdc6eea None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:47.241922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9caa1c-98c5-49a3-a55d-6d20dfdc6eea None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:47.250923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df30fc-b127-4d63-92fd-9f878c35c63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:47.251560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df30fc-b127-4d63-92fd-9f878c35c63a admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:47.253019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df30fc-b127-4d63-92fd-9f878c35c63a admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:47.253180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df30fc-b127-4d63-92fd-9f878c35c63a admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:47.253921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df30fc-b127-4d63-92fd-9f878c35c63a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:47.254238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df30fc-b127-4d63-92fd-9f878c35c63a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:47.255918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df30fc-b127-4d63-92fd-9f878c35c63a admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:47.278200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1134/2218] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:47.279722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df30fc-b127-4d63-92fd-9f878c35c63a admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:47.284350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e58a92-4366-4193-bbcb-9de897486b6a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:47.294957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df30fc-b127-4d63-92fd-9f878c35c63a admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:47.302844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47dd50-9ad6-4fa5-b315-9432e5fbf0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:47.303464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47dd50-9ad6-4fa5-b315-9432e5fbf0ae admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:47.304033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47dd50-9ad6-4fa5-b315-9432e5fbf0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:47.304364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47dd50-9ad6-4fa5-b315-9432e5fbf0ae admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:47.305033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47dd50-9ad6-4fa5-b315-9432e5fbf0ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:47.307126 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47dd50-9ad6-4fa5-b315-9432e5fbf0ae admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:47.308904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:47.312366 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:47.313324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:47.314089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] PATH_INFO: `/v3/projects/3b79d576377941c6b9b1cbaac66d50ac` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:47.320091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b79d576377941c6b9b1cbaac66d50ac)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:47.320091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:47.536210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1135/2219] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:47 2022] POST /identity/v3/projects => generated 383 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:47.536773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47dd50-9ad6-4fa5-b315-9432e5fbf0ae admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:47.627794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b79d576377941c6b9b1cbaac66d50ac', 'request_id': 'req-ddce6a42-bfed-43b1-b010-03a30212e69e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:47.636616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b79d576377941c6b9b1cbaac66d50ac', 'request_id': 'req-ddce6a42-bfed-43b1-b010-03a30212e69e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:47.645413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47dd50-9ad6-4fa5-b315-9432e5fbf0ae admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:47.648232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b79d576377941c6b9b1cbaac66d50ac', 'request_id': 'req-ddce6a42-bfed-43b1-b010-03a30212e69e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:47.749682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] The token cache is being invalidate because project 3b79d576377941c6b9b1cbaac66d50ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:47.749682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:47.749682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:47.749682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1085/2220] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:47 2022] DELETE /identity/v3/projects/3b79d576377941c6b9b1cbaac66d50ac => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:47.749682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:47.751206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:47.760473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e57e76-2601-4722-9567-0c4a5884cd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:47.761759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e57e76-2601-4722-9567-0c4a5884cd03 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:47.762431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e57e76-2601-4722-9567-0c4a5884cd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:47.762767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e57e76-2601-4722-9567-0c4a5884cd03 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:47.764004 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e57e76-2601-4722-9567-0c4a5884cd03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:47.770223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce6a42-bfed-43b1-b010-03a30212e69e admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:47.776547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:47.777981 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:47.778490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:47.778997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] PATH_INFO: `/v3/projects/ba67e453721f465cad36ee25a030c759` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:47.780318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e57e76-2601-4722-9567-0c4a5884cd03 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:47.781080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba67e453721f465cad36ee25a030c759)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:47.783468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:47.963554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1136/2221] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:47 2022] POST /identity/v3/users => generated 292 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:47.965517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e7229-383f-4ca5-8b76-5ddf9ca23c7f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:47.965906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e7229-383f-4ca5-8b76-5ddf9ca23c7f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:47.966894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e7229-383f-4ca5-8b76-5ddf9ca23c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:47.971553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:47.992907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba67e453721f465cad36ee25a030c759', 'request_id': 'req-ec2378e4-3548-4e79-bf81-186cd76d0abf'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:47.995557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-247e7229-383f-4ca5-8b76-5ddf9ca23c7f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:47.996862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-247e7229-383f-4ca5-8b76-5ddf9ca23c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:48.007136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba67e453721f465cad36ee25a030c759', 'request_id': 'req-ec2378e4-3548-4e79-bf81-186cd76d0abf'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:48.020646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba67e453721f465cad36ee25a030c759', 'request_id': 'req-ec2378e4-3548-4e79-bf81-186cd76d0abf'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:48.068336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1137/2222] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:48.068963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e7229-383f-4ca5-8b76-5ddf9ca23c7f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:48.070391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-247e7229-383f-4ca5-8b76-5ddf9ca23c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:48.087719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e7229-383f-4ca5-8b76-5ddf9ca23c7f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:48.096106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a72af3-070b-4b69-a1b7-490c80668dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:48.097762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a72af3-070b-4b69-a1b7-490c80668dcb admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.098223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a72af3-070b-4b69-a1b7-490c80668dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.098844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a72af3-070b-4b69-a1b7-490c80668dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.099958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a72af3-070b-4b69-a1b7-490c80668dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:48.103818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a72af3-070b-4b69-a1b7-490c80668dcb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:48.105786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] The token cache is being invalidate because project ba67e453721f465cad36ee25a030c759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:04:48.106324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:48.107210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ec2378e4-3548-4e79-bf81-186cd76d0abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:04:48.109831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1086/2223] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:04:47 2022] DELETE /identity/v3/projects/ba67e453721f465cad36ee25a030c759 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:48.111666 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4e6b6-bb5b-4ebf-81c3-c479e85b354b None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.112157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4e6b6-bb5b-4ebf-81c3-c479e85b354b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.112630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4e6b6-bb5b-4ebf-81c3-c479e85b354b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.129393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1138/2224] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:48.129393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c432c-79bb-43ff-9f81-76cb586db18a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.129855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c432c-79bb-43ff-9f81-76cb586db18a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.130057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c432c-79bb-43ff-9f81-76cb586db18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.135927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-dae4e6b6-bb5b-4ebf-81c3-c479e85b354b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:48.137389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-dae4e6b6-bb5b-4ebf-81c3-c479e85b354b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:48.152892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1087/2225] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:48.153615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4e6b6-bb5b-4ebf-81c3-c479e85b354b None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:48.155097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-dae4e6b6-bb5b-4ebf-81c3-c479e85b354b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:48.162614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-378c432c-79bb-43ff-9f81-76cb586db18a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:48.164568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-378c432c-79bb-43ff-9f81-76cb586db18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:48.173249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4e6b6-bb5b-4ebf-81c3-c479e85b354b None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:48.184302 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1139/2226] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:48.184302 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c432c-79bb-43ff-9f81-76cb586db18a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:48.195089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21b973-f961-425f-a3ea-388b1c93e46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:48.196088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21b973-f961-425f-a3ea-388b1c93e46b admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.196668 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21b973-f961-425f-a3ea-388b1c93e46b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.198309 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21b973-f961-425f-a3ea-388b1c93e46b admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.198522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c432c-79bb-43ff-9f81-76cb586db18a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:48.198687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21b973-f961-425f-a3ea-388b1c93e46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:48.201779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21b973-f961-425f-a3ea-388b1c93e46b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:48.209270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5042d-1895-47d9-b716-157826ffa978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:48.213076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5042d-1895-47d9-b716-157826ffa978 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.213076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5042d-1895-47d9-b716-157826ffa978 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.213076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5042d-1895-47d9-b716-157826ffa978 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.213750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5042d-1895-47d9-b716-157826ffa978 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:48.214519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5042d-1895-47d9-b716-157826ffa978 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:48.216949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5042d-1895-47d9-b716-157826ffa978 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:48.248529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1088/2227] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:48.248529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc6658-7874-40b3-aa9b-250a43428f9a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.248529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc6658-7874-40b3-aa9b-250a43428f9a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.248529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc6658-7874-40b3-aa9b-250a43428f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.254693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1140/2228] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:48.256812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5042d-1895-47d9-b716-157826ffa978 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:48.279003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5042d-1895-47d9-b716-157826ffa978 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:48.287757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d3509-5ed6-4d96-9bb2-f93a74966434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:48.288163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b2dc6658-7874-40b3-aa9b-250a43428f9a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:48.288877 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d3509-5ed6-4d96-9bb2-f93a74966434 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.289502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d3509-5ed6-4d96-9bb2-f93a74966434 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.290096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d3509-5ed6-4d96-9bb2-f93a74966434 admin admin] PATH_INFO: `/v3/projects/950b28fc6faf434482bbb1c2db3fa194/users/2b8dc16b63584a46b9f63b52aa5ce391/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.291054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b2dc6658-7874-40b3-aa9b-250a43428f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:48.323068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1089/2229] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:48.323068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc6658-7874-40b3-aa9b-250a43428f9a None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:48.329256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d3509-5ed6-4d96-9bb2-f93a74966434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=950b28fc6faf434482bbb1c2db3fa194, user_id=2b8dc16b63584a46b9f63b52aa5ce391, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:48.334742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d3509-5ed6-4d96-9bb2-f93a74966434 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:48.335008 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc6658-7874-40b3-aa9b-250a43428f9a None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:48.340527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dc1cc-0bb3-4aa8-b1dc-1081565b3485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:48.341158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc1cc-0bb3-4aa8-b1dc-1081565b3485 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.341468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc1cc-0bb3-4aa8-b1dc-1081565b3485 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.341699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc1cc-0bb3-4aa8-b1dc-1081565b3485 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.342438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dc1cc-0bb3-4aa8-b1dc-1081565b3485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:48.345717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dc1cc-0bb3-4aa8-b1dc-1081565b3485 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:48.479582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1141/2230] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:48 2022] PUT /identity/v3/projects/950b28fc6faf434482bbb1c2db3fa194/users/2b8dc16b63584a46b9f63b52aa5ce391/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:48.480940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d3509-5ed6-4d96-9bb2-f93a74966434 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:48.541434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1090/2231] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:48 2022] POST /identity/v3/projects => generated 375 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:48.549778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7593-8451-4c18-940f-40030493e044 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.550763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7593-8451-4c18-940f-40030493e044 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.552107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7593-8451-4c18-940f-40030493e044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.606548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d3509-5ed6-4d96-9bb2-f93a74966434 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:48.613637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a2ba7593-8451-4c18-940f-40030493e044 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:48.614979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a2ba7593-8451-4c18-940f-40030493e044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:48.733178 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1091/2232] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:48.745139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba7593-8451-4c18-940f-40030493e044 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:48.751537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0680a-366c-45a4-a3de-628f3f2732c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:48.752921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0680a-366c-45a4-a3de-628f3f2732c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.753361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0680a-366c-45a4-a3de-628f3f2732c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.753788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0680a-366c-45a4-a3de-628f3f2732c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.755238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0680a-366c-45a4-a3de-628f3f2732c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:04:48.755787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0680a-366c-45a4-a3de-628f3f2732c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:48.760750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0680a-366c-45a4-a3de-628f3f2732c9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:48.785343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba7593-8451-4c18-940f-40030493e044 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:48.788875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1142/2233] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:04:48.790098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0680a-366c-45a4-a3de-628f3f2732c9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:48.812346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-54c0680a-366c-45a4-a3de-628f3f2732c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:48.821800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e3586-6551-48cb-92ac-2a1d98e5803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:48.822722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e3586-6551-48cb-92ac-2a1d98e5803d admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.823009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e3586-6551-48cb-92ac-2a1d98e5803d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.823901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e3586-6551-48cb-92ac-2a1d98e5803d admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.824852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e3586-6551-48cb-92ac-2a1d98e5803d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:48.828773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e3586-6551-48cb-92ac-2a1d98e5803d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:48.963054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0680a-366c-45a4-a3de-628f3f2732c9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:48.969011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:48.971534 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ff183-2663-4068-94d5-495e747542d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:48.973417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ff183-2663-4068-94d5-495e747542d8 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:48.974199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ff183-2663-4068-94d5-495e747542d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:48.974780 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ff183-2663-4068-94d5-495e747542d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:48.987405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1092/2234] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:48 2022] POST /identity/v3/users => generated 288 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:48.989570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e3586-6551-48cb-92ac-2a1d98e5803d admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:48.996685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ff183-2663-4068-94d5-495e747542d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:48.999947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ff183-2663-4068-94d5-495e747542d8 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:49.017031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e3586-6551-48cb-92ac-2a1d98e5803d admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:49.032254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1143/2235] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:49.032254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ff183-2663-4068-94d5-495e747542d8 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:49.032604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ea66b-3e2e-4f38-a938-e1b8f3a4559a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:49.032604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ea66b-3e2e-4f38-a938-e1b8f3a4559a admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:49.032604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ea66b-3e2e-4f38-a938-e1b8f3a4559a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:49.032604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ea66b-3e2e-4f38-a938-e1b8f3a4559a admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:49.033206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ea66b-3e2e-4f38-a938-e1b8f3a4559a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:49.073176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ff183-2663-4068-94d5-495e747542d8 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:49.075007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ea66b-3e2e-4f38-a938-e1b8f3a4559a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:49.075190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7070f1-9f4d-4cad-926d-def96f055638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:49.075190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7070f1-9f4d-4cad-926d-def96f055638 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:49.075190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7070f1-9f4d-4cad-926d-def96f055638 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:49.075190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7070f1-9f4d-4cad-926d-def96f055638 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:49.075813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7070f1-9f4d-4cad-926d-def96f055638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:49.075813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7070f1-9f4d-4cad-926d-def96f055638 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:49.080427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1144/2236] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:49.086959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7070f1-9f4d-4cad-926d-def96f055638 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:49.104120 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7070f1-9f4d-4cad-926d-def96f055638 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:49.111684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0455035-e2de-4760-a999-efbab6670bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:49.125115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0455035-e2de-4760-a999-efbab6670bed admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:49.125115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0455035-e2de-4760-a999-efbab6670bed admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:49.125115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0455035-e2de-4760-a999-efbab6670bed admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:49.125115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0455035-e2de-4760-a999-efbab6670bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:49.125115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0455035-e2de-4760-a999-efbab6670bed admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:49.130926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1145/2237] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:04:49.135347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0455035-e2de-4760-a999-efbab6670bed admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:49.146746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0455035-e2de-4760-a999-efbab6670bed admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:49.152958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8c536-6b89-4551-a096-ee82f354eb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:49.153846 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8c536-6b89-4551-a096-ee82f354eb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:49.154207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8c536-6b89-4551-a096-ee82f354eb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:49.154573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8c536-6b89-4551-a096-ee82f354eb5c admin admin] PATH_INFO: `/v3/projects/037b3b78abee48588ee5addef8b70cdc/users/9eb3a5d65f3d4fe08fef3167e5cafb47/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:49.180444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8c536-6b89-4551-a096-ee82f354eb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=037b3b78abee48588ee5addef8b70cdc, user_id=9eb3a5d65f3d4fe08fef3167e5cafb47, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:49.183871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8c536-6b89-4551-a096-ee82f354eb5c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:49.253144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1093/2238] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:48 2022] POST /identity/v3/projects => generated 379 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:49.258541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ea66b-3e2e-4f38-a938-e1b8f3a4559a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:49.419832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ea66b-3e2e-4f38-a938-e1b8f3a4559a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:49.424982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1146/2239] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:49 2022] PUT /identity/v3/projects/037b3b78abee48588ee5addef8b70cdc/users/9eb3a5d65f3d4fe08fef3167e5cafb47/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:49.453184 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f1eb7-880d-4aa3-9862-e4f7dfabb924 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:49.460471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f1eb7-880d-4aa3-9862-e4f7dfabb924 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:49.460471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f1eb7-880d-4aa3-9862-e4f7dfabb924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:49.620600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315921de-1d28-4bc3-8008-eef69086ab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:49.626836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315921de-1d28-4bc3-8008-eef69086ab1f admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:49.631747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315921de-1d28-4bc3-8008-eef69086ab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:49.632768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315921de-1d28-4bc3-8008-eef69086ab1f admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:49.634639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315921de-1d28-4bc3-8008-eef69086ab1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:49.635726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c06f1eb7-880d-4aa3-9862-e4f7dfabb924 None None] MFA Rules not processed for user `9eb3a5d65f3d4fe08fef3167e5cafb47`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:49.644499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315921de-1d28-4bc3-8008-eef69086ab1f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:49.645281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c06f1eb7-880d-4aa3-9862-e4f7dfabb924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:49.782121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1094/2240] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:49 2022] POST /identity/v3/users => generated 290 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:04:49.787450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315921de-1d28-4bc3-8008-eef69086ab1f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:49.817663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315921de-1d28-4bc3-8008-eef69086ab1f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:49.832364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9458a-05ba-46f9-b80b-f995d211be6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:49.834457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9458a-05ba-46f9-b80b-f995d211be6c admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:49.835314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9458a-05ba-46f9-b80b-f995d211be6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:49.836059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9458a-05ba-46f9-b80b-f995d211be6c admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:49.838270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9458a-05ba-46f9-b80b-f995d211be6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:49.843362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9458a-05ba-46f9-b80b-f995d211be6c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:49.851872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1147/2241] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:49.859873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f1eb7-880d-4aa3-9862-e4f7dfabb924 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:49.876756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1095/2242] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:04:49.882334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f1eb7-880d-4aa3-9862-e4f7dfabb924 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:49.891203 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9458a-05ba-46f9-b80b-f995d211be6c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:49.916480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da115b05-2f26-43fc-947f-4619982f2ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:49.917895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da115b05-2f26-43fc-947f-4619982f2ebb admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:49.918486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da115b05-2f26-43fc-947f-4619982f2ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:49.918996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da115b05-2f26-43fc-947f-4619982f2ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:49.920162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da115b05-2f26-43fc-947f-4619982f2ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:49.925888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da115b05-2f26-43fc-947f-4619982f2ebb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:49.937017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9458a-05ba-46f9-b80b-f995d211be6c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:49.949783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf9881-d3e8-4a83-ad4f-155d3d523459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:49.951516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf9881-d3e8-4a83-ad4f-155d3d523459 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:49.954548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf9881-d3e8-4a83-ad4f-155d3d523459 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:49.955241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf9881-d3e8-4a83-ad4f-155d3d523459 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:49.956813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf9881-d3e8-4a83-ad4f-155d3d523459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:49.960980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf9881-d3e8-4a83-ad4f-155d3d523459 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:49.992089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1096/2243] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:49.994780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf9881-d3e8-4a83-ad4f-155d3d523459 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:50.012577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf9881-d3e8-4a83-ad4f-155d3d523459 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:50.019466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fca79b-c9e6-4443-954c-9c2e89ab466b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:50.020917 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fca79b-c9e6-4443-954c-9c2e89ab466b admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:50.021196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fca79b-c9e6-4443-954c-9c2e89ab466b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:50.021674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fca79b-c9e6-4443-954c-9c2e89ab466b admin admin] PATH_INFO: `/v3/projects/d4da0e2bff704d7a944445d0011a8667/users/76771369a0df4c5189e42291ebdf9d24/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:50.052841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fca79b-c9e6-4443-954c-9c2e89ab466b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4da0e2bff704d7a944445d0011a8667, user_id=76771369a0df4c5189e42291ebdf9d24, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:50.056935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fca79b-c9e6-4443-954c-9c2e89ab466b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:50.101277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1148/2244] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:49 2022] POST /identity/v3/projects => generated 377 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:04:50.103407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da115b05-2f26-43fc-947f-4619982f2ebb admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:50.165137 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1097/2245] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:49 2022] PUT /identity/v3/projects/d4da0e2bff704d7a944445d0011a8667/users/76771369a0df4c5189e42291ebdf9d24/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:50.173077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd0e48-051b-4589-9cda-638564ce7582 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:50.173543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd0e48-051b-4589-9cda-638564ce7582 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:50.173997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd0e48-051b-4589-9cda-638564ce7582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:50.220056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f9fd0e48-051b-4589-9cda-638564ce7582 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:50.224513 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f9fd0e48-051b-4589-9cda-638564ce7582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:50.255285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da115b05-2f26-43fc-947f-4619982f2ebb admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:50.393256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86302f93-f375-4067-8aaf-00501e94c202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:50.394022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86302f93-f375-4067-8aaf-00501e94c202 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:50.394294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86302f93-f375-4067-8aaf-00501e94c202 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:50.394625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86302f93-f375-4067-8aaf-00501e94c202 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:50.395297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86302f93-f375-4067-8aaf-00501e94c202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:50.397557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86302f93-f375-4067-8aaf-00501e94c202 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:50.440452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1098/2246] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:50.448089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:50.460876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd0e48-051b-4589-9cda-638564ce7582 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:50.466945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f9fd0e48-051b-4589-9cda-638564ce7582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:50.489743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1149/2247] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:50 2022] POST /identity/v3/users => generated 289 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:50.494274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86302f93-f375-4067-8aaf-00501e94c202 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:50.533012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86302f93-f375-4067-8aaf-00501e94c202 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:50.545382 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90355e5-fa17-4f39-be94-33c09a67aac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:50.546450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90355e5-fa17-4f39-be94-33c09a67aac3 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:50.546888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90355e5-fa17-4f39-be94-33c09a67aac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:50.547283 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90355e5-fa17-4f39-be94-33c09a67aac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:50.548233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90355e5-fa17-4f39-be94-33c09a67aac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:50.550341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90355e5-fa17-4f39-be94-33c09a67aac3 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:50.572787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1150/2248] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:04:50.576812 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90355e5-fa17-4f39-be94-33c09a67aac3 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:50.593631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90355e5-fa17-4f39-be94-33c09a67aac3 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:50.606888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac495e-16dc-4a26-9555-2f6d98e9cee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:50.608691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac495e-16dc-4a26-9555-2f6d98e9cee8 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:50.610433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac495e-16dc-4a26-9555-2f6d98e9cee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:50.611208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac495e-16dc-4a26-9555-2f6d98e9cee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:50.618505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac495e-16dc-4a26-9555-2f6d98e9cee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:50.627364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac495e-16dc-4a26-9555-2f6d98e9cee8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:50.648775 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd0e48-051b-4589-9cda-638564ce7582 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:50.656350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb1b49-9981-4a32-81ae-54d4a58c3565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:50.657570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb1b49-9981-4a32-81ae-54d4a58c3565 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:50.658243 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb1b49-9981-4a32-81ae-54d4a58c3565 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:50.658863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb1b49-9981-4a32-81ae-54d4a58c3565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:50.663624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1151/2249] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:04:50.669933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac495e-16dc-4a26-9555-2f6d98e9cee8 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:50.681187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb1b49-9981-4a32-81ae-54d4a58c3565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:50.683693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb1b49-9981-4a32-81ae-54d4a58c3565 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:50.699432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac495e-16dc-4a26-9555-2f6d98e9cee8 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:50.704229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1099/2250] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:50.710140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c60da-574d-4c1f-8714-13ee28eb2431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:50.715860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c60da-574d-4c1f-8714-13ee28eb2431 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:50.715860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c60da-574d-4c1f-8714-13ee28eb2431 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:50.715860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c60da-574d-4c1f-8714-13ee28eb2431 admin admin] PATH_INFO: `/v3/projects/b303d0dff0cc415cb47202d381752d56/users/376c5d377ea940b8a57fc3fc71c04092/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:50.768148 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c60da-574d-4c1f-8714-13ee28eb2431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b303d0dff0cc415cb47202d381752d56, user_id=376c5d377ea940b8a57fc3fc71c04092, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:50.774751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c60da-574d-4c1f-8714-13ee28eb2431 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:50.891052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1152/2251] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:50 2022] PUT /identity/v3/projects/b303d0dff0cc415cb47202d381752d56/users/376c5d377ea940b8a57fc3fc71c04092/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:04:50.903065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cfce4-a33f-40f6-9ab0-cadcdeaad61e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:50.904711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cfce4-a33f-40f6-9ab0-cadcdeaad61e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:50.905798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cfce4-a33f-40f6-9ab0-cadcdeaad61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:50.976999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-913cfce4-a33f-40f6-9ab0-cadcdeaad61e None None] MFA Rules not processed for user `376c5d377ea940b8a57fc3fc71c04092`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:50.978748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-913cfce4-a33f-40f6-9ab0-cadcdeaad61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:51.183504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1100/2252] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:51.187459 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c60da-574d-4c1f-8714-13ee28eb2431 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:51.339648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c60da-574d-4c1f-8714-13ee28eb2431 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:51.349830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa34e62-ffbf-48f3-b483-281618a768de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:51.349830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa34e62-ffbf-48f3-b483-281618a768de admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:51.350268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa34e62-ffbf-48f3-b483-281618a768de admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:51.350473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa34e62-ffbf-48f3-b483-281618a768de admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:51.351865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa34e62-ffbf-48f3-b483-281618a768de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:51.355861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa34e62-ffbf-48f3-b483-281618a768de admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:51.501701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1153/2253] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:04:51 2022] POST /identity/v3/projects => generated 377 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:51.502586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cfce4-a33f-40f6-9ab0-cadcdeaad61e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:51.524058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cfce4-a33f-40f6-9ab0-cadcdeaad61e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:51.624279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67cf1d-7b77-40dc-94ad-ce15150dec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:51.624907 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67cf1d-7b77-40dc-94ad-ce15150dec6d admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:51.625195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67cf1d-7b77-40dc-94ad-ce15150dec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:51.625529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67cf1d-7b77-40dc-94ad-ce15150dec6d admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:51.626427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67cf1d-7b77-40dc-94ad-ce15150dec6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:51.628886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67cf1d-7b77-40dc-94ad-ce15150dec6d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:51.739036 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1101/2254] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:04:51 2022] POST /identity/v3/users => generated 295 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:04:51.743455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa34e62-ffbf-48f3-b483-281618a768de admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:51.763576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa34e62-ffbf-48f3-b483-281618a768de admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:51.765878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510a1d7-354b-4635-abf6-ec4f0c7d9b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:51.766752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510a1d7-354b-4635-abf6-ec4f0c7d9b1e admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:51.767107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510a1d7-354b-4635-abf6-ec4f0c7d9b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:51.767581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510a1d7-354b-4635-abf6-ec4f0c7d9b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:51.768555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510a1d7-354b-4635-abf6-ec4f0c7d9b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:51.770588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510a1d7-354b-4635-abf6-ec4f0c7d9b1e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:51.789854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1154/2255] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:04:51.794019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67cf1d-7b77-40dc-94ad-ce15150dec6d admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:51.806721 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:51.814152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67cf1d-7b77-40dc-94ad-ce15150dec6d admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:51.819608 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be271a40-bdec-4e8f-8bf9-a581f8544c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:51.820379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be271a40-bdec-4e8f-8bf9-a581f8544c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:51.820659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be271a40-bdec-4e8f-8bf9-a581f8544c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:51.820971 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be271a40-bdec-4e8f-8bf9-a581f8544c1c admin admin] PATH_INFO: `/v3/projects/9a708057745540caacc752a53aceeec0/users/a22e14a97efd46489a605194f6ce3b78/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:51.838755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be271a40-bdec-4e8f-8bf9-a581f8544c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a708057745540caacc752a53aceeec0, user_id=a22e14a97efd46489a605194f6ce3b78, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:51.841849 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be271a40-bdec-4e8f-8bf9-a581f8544c1c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:51.918183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1102/2256] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:51 2022] PUT /identity/v3/projects/9a708057745540caacc752a53aceeec0/users/a22e14a97efd46489a605194f6ce3b78/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:51.924913 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510a1d7-354b-4635-abf6-ec4f0c7d9b1e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:51.943866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510a1d7-354b-4635-abf6-ec4f0c7d9b1e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:51.960987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:52.034239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fffc77-b9de-4996-9744-ebb1b1a1effb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:52.034879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fffc77-b9de-4996-9744-ebb1b1a1effb admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:52.035175 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fffc77-b9de-4996-9744-ebb1b1a1effb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:52.036570 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fffc77-b9de-4996-9744-ebb1b1a1effb admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:52.037281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fffc77-b9de-4996-9744-ebb1b1a1effb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:52.039157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fffc77-b9de-4996-9744-ebb1b1a1effb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:52.053577 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1155/2257] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:04:52.057787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be271a40-bdec-4e8f-8bf9-a581f8544c1c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:52.073456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be271a40-bdec-4e8f-8bf9-a581f8544c1c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:52.083189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cd59a-c096-4d64-bc7c-ee3b4675eb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:52.084614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cd59a-c096-4d64-bc7c-ee3b4675eb3a admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:52.085480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cd59a-c096-4d64-bc7c-ee3b4675eb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:52.086328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cd59a-c096-4d64-bc7c-ee3b4675eb3a admin admin] PATH_INFO: `/v3/projects/9a708057745540caacc752a53aceeec0/users/a22e14a97efd46489a605194f6ce3b78/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:52.096051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cd59a-c096-4d64-bc7c-ee3b4675eb3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a708057745540caacc752a53aceeec0, user_id=a22e14a97efd46489a605194f6ce3b78, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:52.099369 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cd59a-c096-4d64-bc7c-ee3b4675eb3a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:52.125070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1103/2258] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:04:52 2022] PUT /identity/v3/projects/9a708057745540caacc752a53aceeec0/users/a22e14a97efd46489a605194f6ce3b78/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:04:52.135672 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25587cc-a2da-44d4-ad72-6e22c76c13ce None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:52.135968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25587cc-a2da-44d4-ad72-6e22c76c13ce None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:52.136454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25587cc-a2da-44d4-ad72-6e22c76c13ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:52.184091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c25587cc-a2da-44d4-ad72-6e22c76c13ce None None] MFA Rules not processed for user `a22e14a97efd46489a605194f6ce3b78`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:04:52.185717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c25587cc-a2da-44d4-ad72-6e22c76c13ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:52.336209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1156/2259] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:04:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:04:52.348766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cd59a-c096-4d64-bc7c-ee3b4675eb3a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:52.350866 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-861cd59a-c096-4d64-bc7c-ee3b4675eb3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:52.433384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cd59a-c096-4d64-bc7c-ee3b4675eb3a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:52.440257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b21514-84b9-4a69-b2b8-3a6922f37c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:52.441867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b21514-84b9-4a69-b2b8-3a6922f37c5a service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:52.442215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b21514-84b9-4a69-b2b8-3a6922f37c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:52.442832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b21514-84b9-4a69-b2b8-3a6922f37c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:52.589163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b21514-84b9-4a69-b2b8-3a6922f37c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:52.593914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b21514-84b9-4a69-b2b8-3a6922f37c5a service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:52.611841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1104/2260] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:54.542904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:55.421998 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:55.669084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25587cc-a2da-44d4-ad72-6e22c76c13ce None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:55.671897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c25587cc-a2da-44d4-ad72-6e22c76c13ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:04:55.704857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25587cc-a2da-44d4-ad72-6e22c76c13ce None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:55.713937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1152fc-d123-45e3-a622-4441128fdaa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:55.714602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1152fc-d123-45e3-a622-4441128fdaa1 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:55.715731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1152fc-d123-45e3-a622-4441128fdaa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:55.715731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1152fc-d123-45e3-a622-4441128fdaa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:55.870643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1152fc-d123-45e3-a622-4441128fdaa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:55.885735 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1152fc-d123-45e3-a622-4441128fdaa1 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:55.912212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1157/2261] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:04:55.956860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:56.990142 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:58.186500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:58.806244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:04:59.360683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b21514-84b9-4a69-b2b8-3a6922f37c5a service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:04:59.379370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b21514-84b9-4a69-b2b8-3a6922f37c5a service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:04:59.395690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30708bc4-b0ca-4341-9c6c-d08d826627e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:04:59.396400 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30708bc4-b0ca-4341-9c6c-d08d826627e6 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:04:59.396685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30708bc4-b0ca-4341-9c6c-d08d826627e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:04:59.396966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30708bc4-b0ca-4341-9c6c-d08d826627e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:04:59.430006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30708bc4-b0ca-4341-9c6c-d08d826627e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:04:59.431403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30708bc4-b0ca-4341-9c6c-d08d826627e6 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:04:59.461762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1105/2262] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:04:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:05:02.562124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ad44a-d4d6-41fa-9e6f-f99796316e55 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:02.566858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ad44a-d4d6-41fa-9e6f-f99796316e55 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:02.567900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ad44a-d4d6-41fa-9e6f-f99796316e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:02.706067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-a22ad44a-d4d6-41fa-9e6f-f99796316e55 None None] MFA Rules not processed for user `2b8dc16b63584a46b9f63b52aa5ce391`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:02.708793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a22ad44a-d4d6-41fa-9e6f-f99796316e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:03.175164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1158/2263] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 627 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:03.180144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30708bc4-b0ca-4341-9c6c-d08d826627e6 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:03.467702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30708bc4-b0ca-4341-9c6c-d08d826627e6 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:03.483104 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198097f-e211-433d-8fe0-5252d5c92f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:03.484573 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198097f-e211-433d-8fe0-5252d5c92f3a admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:03.485519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198097f-e211-433d-8fe0-5252d5c92f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:03.486269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198097f-e211-433d-8fe0-5252d5c92f3a admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:03.496116 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198097f-e211-433d-8fe0-5252d5c92f3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:03.502612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198097f-e211-433d-8fe0-5252d5c92f3a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:03.744378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1106/2264] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:05:03 2022] POST /identity/v3/projects => generated 381 bytes in 565 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:05:03.748770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ad44a-d4d6-41fa-9e6f-f99796316e55 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:03.809356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ad44a-d4d6-41fa-9e6f-f99796316e55 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:03.956521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ba5fe-9195-449e-be01-f8fa0aae2060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:03.956768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ba5fe-9195-449e-be01-f8fa0aae2060 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:03.957067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ba5fe-9195-449e-be01-f8fa0aae2060 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:03.957346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ba5fe-9195-449e-be01-f8fa0aae2060 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:03.958258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ba5fe-9195-449e-be01-f8fa0aae2060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:03.960522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ba5fe-9195-449e-be01-f8fa0aae2060 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:04.119686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1159/2265] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:05:03 2022] POST /identity/v3/users => generated 297 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:05:04.120922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198097f-e211-433d-8fe0-5252d5c92f3a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:04.149150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198097f-e211-433d-8fe0-5252d5c92f3a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:04.171492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cfaf6-dce4-4921-941f-9a7a651d8c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:04.173717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cfaf6-dce4-4921-941f-9a7a651d8c28 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:04.174211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cfaf6-dce4-4921-941f-9a7a651d8c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:04.176423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cfaf6-dce4-4921-941f-9a7a651d8c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:04.179397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cfaf6-dce4-4921-941f-9a7a651d8c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:04.187438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cfaf6-dce4-4921-941f-9a7a651d8c28 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:04.213280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1107/2266] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:05:04.222664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ba5fe-9195-449e-be01-f8fa0aae2060 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:04.236928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ba5fe-9195-449e-be01-f8fa0aae2060 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:04.251220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158188d8-a960-4b27-b467-02ce73892ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:04.251934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158188d8-a960-4b27-b467-02ce73892ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:04.252547 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158188d8-a960-4b27-b467-02ce73892ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:04.253155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158188d8-a960-4b27-b467-02ce73892ca8 admin admin] PATH_INFO: `/v3/projects/1430f7af3a724f22b7bc770393490635/users/53d15c73206044a2a1d7418587018653/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:04.305570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d95ec-c456-48c6-8674-80dc168e0979 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:04.305857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d95ec-c456-48c6-8674-80dc168e0979 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:04.306294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d95ec-c456-48c6-8674-80dc168e0979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:04.325483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158188d8-a960-4b27-b467-02ce73892ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1430f7af3a724f22b7bc770393490635, user_id=53d15c73206044a2a1d7418587018653, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:04.337779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158188d8-a960-4b27-b467-02ce73892ca8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:04.390257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d05d95ec-c456-48c6-8674-80dc168e0979 None None] MFA Rules not processed for user `76771369a0df4c5189e42291ebdf9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:04.391865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d05d95ec-c456-48c6-8674-80dc168e0979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:04.495052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:04.512919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1160/2267] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:05:04 2022] PUT /identity/v3/projects/1430f7af3a724f22b7bc770393490635/users/53d15c73206044a2a1d7418587018653/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:04.524780 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158188d8-a960-4b27-b467-02ce73892ca8 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:04.525799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1108/2268] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:05:04.552710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d95ec-c456-48c6-8674-80dc168e0979 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:04.583861 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:04.626002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d95ec-c456-48c6-8674-80dc168e0979 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:04.729475 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158188d8-a960-4b27-b467-02ce73892ca8 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:04.753496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c9ece-68ae-4b8a-8b98-275eab56aa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:04.756021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c9ece-68ae-4b8a-8b98-275eab56aa74 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:04.759090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c9ece-68ae-4b8a-8b98-275eab56aa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:04.759943 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c9ece-68ae-4b8a-8b98-275eab56aa74 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:04.761674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c9ece-68ae-4b8a-8b98-275eab56aa74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:04.769984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c9ece-68ae-4b8a-8b98-275eab56aa74 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:04.823404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1161/2269] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:05:04.838756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c9ece-68ae-4b8a-8b98-275eab56aa74 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:04.891023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c9ece-68ae-4b8a-8b98-275eab56aa74 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:04.917467 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deaf392-309f-4b7f-910f-c0a1cc9fa242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:04.918394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaf392-309f-4b7f-910f-c0a1cc9fa242 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:04.918932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaf392-309f-4b7f-910f-c0a1cc9fa242 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:04.919533 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deaf392-309f-4b7f-910f-c0a1cc9fa242 admin admin] PATH_INFO: `/v3/projects/1430f7af3a724f22b7bc770393490635/users/53d15c73206044a2a1d7418587018653/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:04.930491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deaf392-309f-4b7f-910f-c0a1cc9fa242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1430f7af3a724f22b7bc770393490635, user_id=53d15c73206044a2a1d7418587018653, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:04.934222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deaf392-309f-4b7f-910f-c0a1cc9fa242 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:04.951745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaa541-953e-424a-a7d5-9a21d3da1242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:04.952825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa541-953e-424a-a7d5-9a21d3da1242 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:04.956743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1162/2270] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:05:04 2022] PUT /identity/v3/projects/1430f7af3a724f22b7bc770393490635/users/53d15c73206044a2a1d7418587018653/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:04.961380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa541-953e-424a-a7d5-9a21d3da1242 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:04.961828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa541-953e-424a-a7d5-9a21d3da1242 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:04.962711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaa541-953e-424a-a7d5-9a21d3da1242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:04.965610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaa541-953e-424a-a7d5-9a21d3da1242 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:05.167398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1109/2271] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:05:04 2022] POST /identity/v3/projects => generated 379 bytes in 616 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:05:05.173710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deaf392-309f-4b7f-910f-c0a1cc9fa242 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:05.293244 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deaf392-309f-4b7f-910f-c0a1cc9fa242 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:05.464740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8bf14-3a62-4cb1-90e6-98f4a1b1e7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:05.465609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8bf14-3a62-4cb1-90e6-98f4a1b1e7ba admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:05.465946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8bf14-3a62-4cb1-90e6-98f4a1b1e7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:05.466484 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8bf14-3a62-4cb1-90e6-98f4a1b1e7ba admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:05.467866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8bf14-3a62-4cb1-90e6-98f4a1b1e7ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:05.470017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8bf14-3a62-4cb1-90e6-98f4a1b1e7ba admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:05.698903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1163/2272] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:05:05 2022] POST /identity/v3/users => generated 296 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:05:05.704862 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaa541-953e-424a-a7d5-9a21d3da1242 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:05.727474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaa541-953e-424a-a7d5-9a21d3da1242 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:05.750978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c9372-bf16-426a-a64b-6f03cc6aba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:05.753720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9372-bf16-426a-a64b-6f03cc6aba0f admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:05.754315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9372-bf16-426a-a64b-6f03cc6aba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:05.755841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9372-bf16-426a-a64b-6f03cc6aba0f admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:05.756780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c9372-bf16-426a-a64b-6f03cc6aba0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:05.763583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c9372-bf16-426a-a64b-6f03cc6aba0f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:05.807892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1110/2273] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:05:05.808952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8bf14-3a62-4cb1-90e6-98f4a1b1e7ba admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:05.837290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a93e5-9433-4a7b-9a46-b5717beaf7f8 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:05.837290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a93e5-9433-4a7b-9a46-b5717beaf7f8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:05.837290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a93e5-9433-4a7b-9a46-b5717beaf7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:05.845996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8bf14-3a62-4cb1-90e6-98f4a1b1e7ba admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:05.877680 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7d4a93e5-9433-4a7b-9a46-b5717beaf7f8 None None] MFA Rules not processed for user `ee6a9052a75a4d8c9f4a08638685012b`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:05.877911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c95428-ed56-42af-8942-73da6a6ac17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:05.879085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95428-ed56-42af-8942-73da6a6ac17e admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:05.879597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95428-ed56-42af-8942-73da6a6ac17e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:05.880108 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95428-ed56-42af-8942-73da6a6ac17e admin admin] PATH_INFO: `/v3/projects/967915d2bb87499d9b15d7f694f76e67/users/e133231fd3264083a2184b9685f212d9/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:05.880664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7d4a93e5-9433-4a7b-9a46-b5717beaf7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:05.930750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c95428-ed56-42af-8942-73da6a6ac17e admin admin] RBAC: Authorizing `identity:create_grant(project_id=967915d2bb87499d9b15d7f694f76e67, user_id=e133231fd3264083a2184b9685f212d9, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:05.936947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c95428-ed56-42af-8942-73da6a6ac17e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:05.994085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1111/2274] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:05:06.074471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1164/2275] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:05:05 2022] PUT /identity/v3/projects/967915d2bb87499d9b15d7f694f76e67/users/e133231fd3264083a2184b9685f212d9/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:05:06.081256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a93e5-9433-4a7b-9a46-b5717beaf7f8 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:06.134600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a93e5-9433-4a7b-9a46-b5717beaf7f8 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:06.347960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34930ab-46d1-434d-880a-c80ed52fe54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:06.349363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34930ab-46d1-434d-880a-c80ed52fe54e admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:06.350028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34930ab-46d1-434d-880a-c80ed52fe54e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:06.350895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34930ab-46d1-434d-880a-c80ed52fe54e admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:06.352509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34930ab-46d1-434d-880a-c80ed52fe54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:06.357468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34930ab-46d1-434d-880a-c80ed52fe54e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:06.387012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1112/2276] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:05:06.391312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c95428-ed56-42af-8942-73da6a6ac17e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:06.424800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c95428-ed56-42af-8942-73da6a6ac17e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:06.442586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71e865-78d6-4bc2-910b-e100ec3c66dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:06.444005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71e865-78d6-4bc2-910b-e100ec3c66dc admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:06.444651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71e865-78d6-4bc2-910b-e100ec3c66dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:06.445299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71e865-78d6-4bc2-910b-e100ec3c66dc admin admin] PATH_INFO: `/v3/projects/967915d2bb87499d9b15d7f694f76e67/users/e133231fd3264083a2184b9685f212d9/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:06.449280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71e865-78d6-4bc2-910b-e100ec3c66dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=967915d2bb87499d9b15d7f694f76e67, user_id=e133231fd3264083a2184b9685f212d9, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:06.454793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71e865-78d6-4bc2-910b-e100ec3c66dc admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:06.480268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1165/2277] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:05:06 2022] PUT /identity/v3/projects/967915d2bb87499d9b15d7f694f76e67/users/e133231fd3264083a2184b9685f212d9/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:05:07.117381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:09.360766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34930ab-46d1-434d-880a-c80ed52fe54e admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:09.379051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34930ab-46d1-434d-880a-c80ed52fe54e admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:09.401961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:09.501579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:09.502772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:09.503168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:09.503545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] PATH_INFO: `/v3/users/9eb3a5d65f3d4fe08fef3167e5cafb47` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:09.505214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb3a5d65f3d4fe08fef3167e5cafb47)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:09.507558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:09.871373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb3a5d65f3d4fe08fef3167e5cafb47'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:09.881963 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb3a5d65f3d4fe08fef3167e5cafb47'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:09.882893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb3a5d65f3d4fe08fef3167e5cafb47'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:09.987397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb3a5d65f3d4fe08fef3167e5cafb47'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:10.025448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb3a5d65f3d4fe08fef3167e5cafb47'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:10.032983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1113/2278] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:05:09 2022] DELETE /identity/v3/users/9eb3a5d65f3d4fe08fef3167e5cafb47 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:10.038256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc71035-b494-4f3a-9da9-3dc3515521a9 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:10.039876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc71035-b494-4f3a-9da9-3dc3515521a9 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:10.040691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc71035-b494-4f3a-9da9-3dc3515521a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:10.069994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9dc71035-b494-4f3a-9da9-3dc3515521a9 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:10.072349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9dc71035-b494-4f3a-9da9-3dc3515521a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:10.161051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1166/2279] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:10.187800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:10.190058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:10.434458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7935767-e4b6-4712-8716-4df2080f7aad admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:10.441081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d1b3c-fbd7-446f-b3c9-80ebf7dc5f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:10.443004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d1b3c-fbd7-446f-b3c9-80ebf7dc5f49 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:10.443004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d1b3c-fbd7-446f-b3c9-80ebf7dc5f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:10.443004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d1b3c-fbd7-446f-b3c9-80ebf7dc5f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:10.456379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d1b3c-fbd7-446f-b3c9-80ebf7dc5f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:10.458358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d1b3c-fbd7-446f-b3c9-80ebf7dc5f49 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:10.476890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1114/2280] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:05:11.733163 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc71035-b494-4f3a-9da9-3dc3515521a9 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:11.734991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9dc71035-b494-4f3a-9da9-3dc3515521a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:11.750239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc71035-b494-4f3a-9da9-3dc3515521a9 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:11.763531 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:11.764260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:11.764564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:11.764845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] PATH_INFO: `/v3/projects/037b3b78abee48588ee5addef8b70cdc` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:11.768979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=037b3b78abee48588ee5addef8b70cdc)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:11.774007 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:12.024379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '037b3b78abee48588ee5addef8b70cdc', 'request_id': 'req-0b840553-48bc-4241-9ba1-e9aae9de0bba'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:12.039816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '037b3b78abee48588ee5addef8b70cdc', 'request_id': 'req-0b840553-48bc-4241-9ba1-e9aae9de0bba'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:12.175338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '037b3b78abee48588ee5addef8b70cdc', 'request_id': 'req-0b840553-48bc-4241-9ba1-e9aae9de0bba'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:12.201495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] The token cache is being invalidate because project 037b3b78abee48588ee5addef8b70cdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:05:12.201803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:12.202604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:12.205369 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1167/2281] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:05:11 2022] DELETE /identity/v3/projects/037b3b78abee48588ee5addef8b70cdc => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:12.209704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d1b3c-fbd7-446f-b3c9-80ebf7dc5f49 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:12.225549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d1b3c-fbd7-446f-b3c9-80ebf7dc5f49 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:12.293755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:12.381877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:12.382890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:12.388300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:12.388300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] PATH_INFO: `/v3/users/376c5d377ea940b8a57fc3fc71c04092` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:12.390748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] RBAC: Authorizing `identity:delete_user(user_id=376c5d377ea940b8a57fc3fc71c04092)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:12.393165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:12.748291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '376c5d377ea940b8a57fc3fc71c04092'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:12.771739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '376c5d377ea940b8a57fc3fc71c04092'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:12.774029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '376c5d377ea940b8a57fc3fc71c04092'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:12.978191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '376c5d377ea940b8a57fc3fc71c04092'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:13.022474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '376c5d377ea940b8a57fc3fc71c04092'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:13.073484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1115/2282] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:05:12 2022] DELETE /identity/v3/users/376c5d377ea940b8a57fc3fc71c04092 => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:15.010823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:15.013639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:15.083985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840553-48bc-4241-9ba1-e9aae9de0bba admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:15.308779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:15.311435 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:15.312263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:15.312959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] PATH_INFO: `/v3/projects/b303d0dff0cc415cb47202d381752d56` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:15.316813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b303d0dff0cc415cb47202d381752d56)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:15.322480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:15.443166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:15.672089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b303d0dff0cc415cb47202d381752d56', 'request_id': 'req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:15.686742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b303d0dff0cc415cb47202d381752d56', 'request_id': 'req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:15.835778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b303d0dff0cc415cb47202d381752d56', 'request_id': 'req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:15.876716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] The token cache is being invalidate because project b303d0dff0cc415cb47202d381752d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:05:15.878623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:15.882431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-9c16a8d0-c67a-4d20-b5b1-8afe81037eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:15.889411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1168/2283] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:05:15 2022] DELETE /identity/v3/projects/b303d0dff0cc415cb47202d381752d56 => generated 0 bytes in 880 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:15.895445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:15.898965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:15.969761 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bce6b9-f857-4dbe-8428-fb3970cb951c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:16.165540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:16.166141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:16.166431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:16.166707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] PATH_INFO: `/v3/users/a22e14a97efd46489a605194f6ce3b78` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:16.168010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22e14a97efd46489a605194f6ce3b78)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:16.170675 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:16.703419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22e14a97efd46489a605194f6ce3b78'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:16.703419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:16.708857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22e14a97efd46489a605194f6ce3b78'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:16.710456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22e14a97efd46489a605194f6ce3b78'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:16.861336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0bb21-cd18-4216-ac74-a2b0a905a9fc None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:16.862857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0bb21-cd18-4216-ac74-a2b0a905a9fc None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:16.863765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0bb21-cd18-4216-ac74-a2b0a905a9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:16.875615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22e14a97efd46489a605194f6ce3b78'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:16.914756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-86b0bb21-cd18-4216-ac74-a2b0a905a9fc None None] MFA Rules not processed for user `0e8d03fe51534f8e867d0f83c3db2d36`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:16.917519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-86b0bb21-cd18-4216-ac74-a2b0a905a9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:16.921962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22e14a97efd46489a605194f6ce3b78'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:16.940279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1116/2284] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:05:15 2022] DELETE /identity/v3/users/a22e14a97efd46489a605194f6ce3b78 => generated 0 bytes in 1047 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:05:17.063982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1169/2285] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:05:18.865825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:18.869682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:19.155691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8b45e-1fbf-49fe-b2ea-c42460e9d0e5 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:19.194028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:19.195537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:19.196317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:19.197043 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] PATH_INFO: `/v3/projects/9a708057745540caacc752a53aceeec0` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:19.200887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a708057745540caacc752a53aceeec0)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:19.207133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:19.489990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a708057745540caacc752a53aceeec0', 'request_id': 'req-0802f4f4-6a59-416b-88c0-b2e285c2177c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:19.530877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a708057745540caacc752a53aceeec0', 'request_id': 'req-0802f4f4-6a59-416b-88c0-b2e285c2177c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:19.563746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a708057745540caacc752a53aceeec0', 'request_id': 'req-0802f4f4-6a59-416b-88c0-b2e285c2177c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:19.722486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] The token cache is being invalidate because project 9a708057745540caacc752a53aceeec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:05:19.722794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:19.723599 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0802f4f4-6a59-416b-88c0-b2e285c2177c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:19.739165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1117/2286] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:05:18 2022] DELETE /identity/v3/projects/9a708057745540caacc752a53aceeec0 => generated 0 bytes in 874 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:19.750976 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d5166-eac1-4041-83b2-ca7bfee62167 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:19.751752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d5166-eac1-4041-83b2-ca7bfee62167 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:19.752987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d5166-eac1-4041-83b2-ca7bfee62167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:19.886601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e54d5166-eac1-4041-83b2-ca7bfee62167 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:19.895320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e54d5166-eac1-4041-83b2-ca7bfee62167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:20.171746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1170/2287] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:20.178968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be5c5f-79dd-46eb-ae03-2dba45f42234 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:20.181756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be5c5f-79dd-46eb-ae03-2dba45f42234 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:20.182488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be5c5f-79dd-46eb-ae03-2dba45f42234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:20.233157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-50be5c5f-79dd-46eb-ae03-2dba45f42234 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:20.234718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-50be5c5f-79dd-46eb-ae03-2dba45f42234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:20.314061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1118/2288] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:20.314242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d5166-eac1-4041-83b2-ca7bfee62167 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:20.326239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d5166-eac1-4041-83b2-ca7bfee62167 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:20.336461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2ff04-e4c8-4cb1-933c-df96ef8c7622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:20.338631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2ff04-e4c8-4cb1-933c-df96ef8c7622 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:20.339579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2ff04-e4c8-4cb1-933c-df96ef8c7622 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:20.340375 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2ff04-e4c8-4cb1-933c-df96ef8c7622 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:20.342464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2ff04-e4c8-4cb1-933c-df96ef8c7622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:05:20.343384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2ff04-e4c8-4cb1-933c-df96ef8c7622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:20.347224 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2ff04-e4c8-4cb1-933c-df96ef8c7622 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:20.382597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1171/2289] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:05:20.503811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be5c5f-79dd-46eb-ae03-2dba45f42234 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:20.539703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be5c5f-79dd-46eb-ae03-2dba45f42234 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:20.560840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea750052-3c21-41a7-b8a7-cff6eb681679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:20.565055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea750052-3c21-41a7-b8a7-cff6eb681679 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:20.566916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea750052-3c21-41a7-b8a7-cff6eb681679 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:20.568729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea750052-3c21-41a7-b8a7-cff6eb681679 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:20.569554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea750052-3c21-41a7-b8a7-cff6eb681679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:20.574944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea750052-3c21-41a7-b8a7-cff6eb681679 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:20.881617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1119/2290] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:05:20 2022] POST /identity/v3/projects => generated 385 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:05:20.888481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2ff04-e4c8-4cb1-933c-df96ef8c7622 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:20.905840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2ff04-e4c8-4cb1-933c-df96ef8c7622 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:21.149274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:21.319263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca4b54-817d-4da3-9cdf-7d40fa0a9ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:21.319263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca4b54-817d-4da3-9cdf-7d40fa0a9ccd admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:21.319263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca4b54-817d-4da3-9cdf-7d40fa0a9ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:21.319263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca4b54-817d-4da3-9cdf-7d40fa0a9ccd admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:21.320139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca4b54-817d-4da3-9cdf-7d40fa0a9ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:21.323045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca4b54-817d-4da3-9cdf-7d40fa0a9ccd admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:21.662032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea750052-3c21-41a7-b8a7-cff6eb681679 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:21.696594 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1172/2291] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:05:20 2022] POST /identity/v3/users => generated 293 bytes in 806 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:05:21.698829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea750052-3c21-41a7-b8a7-cff6eb681679 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:21.717315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb4a69-9a5c-471b-a411-34b8d076db7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:21.720837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb4a69-9a5c-471b-a411-34b8d076db7c admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:21.722105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb4a69-9a5c-471b-a411-34b8d076db7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:21.725103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb4a69-9a5c-471b-a411-34b8d076db7c admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:21.736594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb4a69-9a5c-471b-a411-34b8d076db7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:21.736594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb4a69-9a5c-471b-a411-34b8d076db7c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:21.772861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1120/2292] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:05:21.774558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca4b54-817d-4da3-9cdf-7d40fa0a9ccd admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:21.818510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca4b54-817d-4da3-9cdf-7d40fa0a9ccd admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:21.843604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e804af-fa59-4c5a-ab3f-4bdfb7ae358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:21.854098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e804af-fa59-4c5a-ab3f-4bdfb7ae358f admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:21.854644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e804af-fa59-4c5a-ab3f-4bdfb7ae358f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:21.856203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e804af-fa59-4c5a-ab3f-4bdfb7ae358f admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:21.857595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e804af-fa59-4c5a-ab3f-4bdfb7ae358f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:21.862218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e804af-fa59-4c5a-ab3f-4bdfb7ae358f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:21.901833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1173/2293] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:05:21.908568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb4a69-9a5c-471b-a411-34b8d076db7c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:21.938353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb4a69-9a5c-471b-a411-34b8d076db7c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:21.973441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b23b9-a099-45a4-a8e5-ff9ece4c1df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:21.977393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b23b9-a099-45a4-a8e5-ff9ece4c1df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:21.980104 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b23b9-a099-45a4-a8e5-ff9ece4c1df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:21.982670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b23b9-a099-45a4-a8e5-ff9ece4c1df1 admin admin] PATH_INFO: `/v3/projects/de6a8daec9374e69a5d35e3ab474a0f6/users/05865668aa914178ad7ea63671a05fb2/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:22.119379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b23b9-a099-45a4-a8e5-ff9ece4c1df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de6a8daec9374e69a5d35e3ab474a0f6, user_id=05865668aa914178ad7ea63671a05fb2, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:22.134600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b23b9-a099-45a4-a8e5-ff9ece4c1df1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:22.154543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:22.257264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e804af-fa59-4c5a-ab3f-4bdfb7ae358f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:22.260704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c6e804af-fa59-4c5a-ab3f-4bdfb7ae358f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:22.365781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e804af-fa59-4c5a-ab3f-4bdfb7ae358f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:22.371079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1121/2294] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:05:21 2022] PUT /identity/v3/projects/de6a8daec9374e69a5d35e3ab474a0f6/users/05865668aa914178ad7ea63671a05fb2/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:22.374434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c470a-fad8-41c3-ab7c-9c45acced0ff None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:22.379293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c470a-fad8-41c3-ab7c-9c45acced0ff None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:22.379293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c470a-fad8-41c3-ab7c-9c45acced0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:22.463872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f11c470a-fad8-41c3-ab7c-9c45acced0ff None None] MFA Rules not processed for user `05865668aa914178ad7ea63671a05fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:22.465217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f11c470a-fad8-41c3-ab7c-9c45acced0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:22.571012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:22.572110 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:22.572661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:22.573154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] PATH_INFO: `/v3/users/69e539a5ddf04791b78a5d2e68899e7b` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:22.575745 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e539a5ddf04791b78a5d2e68899e7b)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:22.580429 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:22.775335 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1122/2295] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:22.844851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c470a-fad8-41c3-ab7c-9c45acced0ff None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:22.852284 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f11c470a-fad8-41c3-ab7c-9c45acced0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:23.073968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e539a5ddf04791b78a5d2e68899e7b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:23.075335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e539a5ddf04791b78a5d2e68899e7b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:23.088949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e539a5ddf04791b78a5d2e68899e7b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:23.092258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e539a5ddf04791b78a5d2e68899e7b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:23.150283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c470a-fad8-41c3-ab7c-9c45acced0ff None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:23.165518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7bd5a-3391-474d-92c4-874ce15a307f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:23.166186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7bd5a-3391-474d-92c4-874ce15a307f service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:23.166472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7bd5a-3391-474d-92c4-874ce15a307f service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:23.166746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7bd5a-3391-474d-92c4-874ce15a307f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:23.171614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b8d7bd5a-3391-474d-92c4-874ce15a307f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:23.206650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7bd5a-3391-474d-92c4-874ce15a307f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:23.216871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7bd5a-3391-474d-92c4-874ce15a307f service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:23.241497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1123/2296] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:05:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:05:23.250044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a352033-2c05-4a9b-ad39-43d60d184dcd None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:23.251207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a352033-2c05-4a9b-ad39-43d60d184dcd None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:23.251481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a352033-2c05-4a9b-ad39-43d60d184dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:23.296922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-944f29b2-beeb-4c6c-a7fc-c4b0c81fbe95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e539a5ddf04791b78a5d2e68899e7b'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:23.361194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1174/2297] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:05:22 2022] DELETE /identity/v3/users/69e539a5ddf04791b78a5d2e68899e7b => generated 0 bytes in 1105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:23.368430 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-1a352033-2c05-4a9b-ad39-43d60d184dcd None None] MFA Rules not processed for user `53d15c73206044a2a1d7418587018653`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:23.369731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1a352033-2c05-4a9b-ad39-43d60d184dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:23.374423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1649d12-5c00-48ef-9c39-bd882e40d79c None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:23.376795 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1649d12-5c00-48ef-9c39-bd882e40d79c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:23.379403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1649d12-5c00-48ef-9c39-bd882e40d79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:23.465820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-c1649d12-5c00-48ef-9c39-bd882e40d79c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:23.471251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c1649d12-5c00-48ef-9c39-bd882e40d79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:23.691209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1124/2298] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:23.703864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a352033-2c05-4a9b-ad39-43d60d184dcd None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:23.710629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1a352033-2c05-4a9b-ad39-43d60d184dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:23.764855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1175/2299] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:23.772974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1649d12-5c00-48ef-9c39-bd882e40d79c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:23.774875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c1649d12-5c00-48ef-9c39-bd882e40d79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:23.939023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a352033-2c05-4a9b-ad39-43d60d184dcd None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:23.961642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462d738-c2b8-44d0-9f2e-cbf43d065a2b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454fb811f6e34e1d881d4948ede058b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:23.963317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462d738-c2b8-44d0-9f2e-cbf43d065a2b service glance] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:23.966905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462d738-c2b8-44d0-9f2e-cbf43d065a2b service glance] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:23.969153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462d738-c2b8-44d0-9f2e-cbf43d065a2b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:23.977729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f462d738-c2b8-44d0-9f2e-cbf43d065a2b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:24.051830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1649d12-5c00-48ef-9c39-bd882e40d79c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:24.072016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cabab3-ed62-455b-979b-d116ae942bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:24.072477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cabab3-ed62-455b-979b-d116ae942bbb service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:24.073008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cabab3-ed62-455b-979b-d116ae942bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:24.073499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cabab3-ed62-455b-979b-d116ae942bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:24.124479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462d738-c2b8-44d0-9f2e-cbf43d065a2b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:24.128731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462d738-c2b8-44d0-9f2e-cbf43d065a2b service glance] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:24.129721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cabab3-ed62-455b-979b-d116ae942bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:24.131880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cabab3-ed62-455b-979b-d116ae942bbb service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:24.160059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1176/2300] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:05:24.183435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1125/2301] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:05:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:05:24.884985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b49e24-8ab8-450c-83b7-95a989fa1b78 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:24.891691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b49e24-8ab8-450c-83b7-95a989fa1b78 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:24.891691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b49e24-8ab8-450c-83b7-95a989fa1b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:24.982767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-43b49e24-8ab8-450c-83b7-95a989fa1b78 None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:24.986832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-43b49e24-8ab8-450c-83b7-95a989fa1b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:25.214966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1177/2302] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:05:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:05:25.288580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462d738-c2b8-44d0-9f2e-cbf43d065a2b service glance] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:25.288580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f462d738-c2b8-44d0-9f2e-cbf43d065a2b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:25.516505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462d738-c2b8-44d0-9f2e-cbf43d065a2b service glance] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:25.544332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26847210-87df-444c-b929-f3ec0697a913 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:25.544968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b49e24-8ab8-450c-83b7-95a989fa1b78 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:25.546759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-43b49e24-8ab8-450c-83b7-95a989fa1b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:25.548536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26847210-87df-444c-b929-f3ec0697a913 service swift] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:25.551831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26847210-87df-444c-b929-f3ec0697a913 service swift] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:25.552896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26847210-87df-444c-b929-f3ec0697a913 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:25.584946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b49e24-8ab8-450c-83b7-95a989fa1b78 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:25.585942 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26847210-87df-444c-b929-f3ec0697a913 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:25.589877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26847210-87df-444c-b929-f3ec0697a913 service swift] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:25.593395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:25.594331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:25.594790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:25.595256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] PATH_INFO: `/v3/projects/7317f7960dc7424fac44883851792b08` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:25.597368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7317f7960dc7424fac44883851792b08)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:25.599803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:25.615543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1126/2303] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:05:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:05:25.827692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7317f7960dc7424fac44883851792b08', 'request_id': 'req-a01af42b-703a-4274-a739-ac6637141f56'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:25.848673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7317f7960dc7424fac44883851792b08', 'request_id': 'req-a01af42b-703a-4274-a739-ac6637141f56'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:25.971788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7317f7960dc7424fac44883851792b08', 'request_id': 'req-a01af42b-703a-4274-a739-ac6637141f56'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:25.994902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] The token cache is being invalidate because project 7317f7960dc7424fac44883851792b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:05:25.995509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:25.997247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a01af42b-703a-4274-a739-ac6637141f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:25.999612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1178/2304] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:05:25 2022] DELETE /identity/v3/projects/7317f7960dc7424fac44883851792b08 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:05:26.007639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49345998-dba8-4962-bf4a-ca25b6c31000 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:26.007639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49345998-dba8-4962-bf4a-ca25b6c31000 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:26.007639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49345998-dba8-4962-bf4a-ca25b6c31000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:26.139262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-49345998-dba8-4962-bf4a-ca25b6c31000 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:26.152040 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-49345998-dba8-4962-bf4a-ca25b6c31000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:26.393093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1127/2305] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:26.394289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e9318-fbb3-43d7-8872-335b931eee91 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:26.394777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e9318-fbb3-43d7-8872-335b931eee91 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:26.395171 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e9318-fbb3-43d7-8872-335b931eee91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:26.474771 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-568e9318-fbb3-43d7-8872-335b931eee91 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:26.482899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-568e9318-fbb3-43d7-8872-335b931eee91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:26.537908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1179/2306] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:26.554844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49345998-dba8-4962-bf4a-ca25b6c31000 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:26.572370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49345998-dba8-4962-bf4a-ca25b6c31000 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:26.588066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f219dd-8f73-40f7-8b22-7b21db7400ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:26.589597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f219dd-8f73-40f7-8b22-7b21db7400ce admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:26.590199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f219dd-8f73-40f7-8b22-7b21db7400ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:26.591672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f219dd-8f73-40f7-8b22-7b21db7400ce admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:26.593192 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f219dd-8f73-40f7-8b22-7b21db7400ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:05:26.593757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f219dd-8f73-40f7-8b22-7b21db7400ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:26.597747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f219dd-8f73-40f7-8b22-7b21db7400ce admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:26.806811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1128/2307] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:05:26.807789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e9318-fbb3-43d7-8872-335b931eee91 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:26.822066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e9318-fbb3-43d7-8872-335b931eee91 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:26.857993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e7187-161d-4cd1-a460-597c8230ea48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:26.858715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e7187-161d-4cd1-a460-597c8230ea48 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:26.859039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e7187-161d-4cd1-a460-597c8230ea48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:26.859347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e7187-161d-4cd1-a460-597c8230ea48 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:26.860156 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e7187-161d-4cd1-a460-597c8230ea48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:26.862623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e7187-161d-4cd1-a460-597c8230ea48 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:27.376500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1180/2308] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:05:26 2022] POST /identity/v3/projects => generated 365 bytes in 570 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:05:27.388472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f219dd-8f73-40f7-8b22-7b21db7400ce admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:27.389867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:27.402800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f219dd-8f73-40f7-8b22-7b21db7400ce admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:27.540874 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6732f58-c0eb-4baa-9a83-03a7bca6259f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:27.542152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6732f58-c0eb-4baa-9a83-03a7bca6259f admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:27.542603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6732f58-c0eb-4baa-9a83-03a7bca6259f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:27.543041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6732f58-c0eb-4baa-9a83-03a7bca6259f admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:27.543783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6732f58-c0eb-4baa-9a83-03a7bca6259f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:27.552815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6732f58-c0eb-4baa-9a83-03a7bca6259f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:27.723796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1129/2309] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:05:27 2022] POST /identity/v3/users => generated 283 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:05:27.725747 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e7187-161d-4cd1-a460-597c8230ea48 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:27.782241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e7187-161d-4cd1-a460-597c8230ea48 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:27.797487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0369099-1359-41c3-9253-3afc14aa5ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:27.799768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0369099-1359-41c3-9253-3afc14aa5ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:27.801612 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0369099-1359-41c3-9253-3afc14aa5ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:27.802236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0369099-1359-41c3-9253-3afc14aa5ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:27.804449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0369099-1359-41c3-9253-3afc14aa5ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:27.810932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0369099-1359-41c3-9253-3afc14aa5ed8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:27.902341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1181/2310] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:05:27.903578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6732f58-c0eb-4baa-9a83-03a7bca6259f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:27.932549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6732f58-c0eb-4baa-9a83-03a7bca6259f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:27.962388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72511325-1a9f-48f0-81f3-78a2d81f3d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:27.966418 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72511325-1a9f-48f0-81f3-78a2d81f3d58 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:27.967832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72511325-1a9f-48f0-81f3-78a2d81f3d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:27.968961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72511325-1a9f-48f0-81f3-78a2d81f3d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:28.007217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72511325-1a9f-48f0-81f3-78a2d81f3d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:28.027490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72511325-1a9f-48f0-81f3-78a2d81f3d58 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:28.123263 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1130/2311] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:05:28.126297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0369099-1359-41c3-9253-3afc14aa5ed8 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:28.163774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0369099-1359-41c3-9253-3afc14aa5ed8 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:28.190224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cd121-b565-4b0e-a346-1db365cbdfa0 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:28.191054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cd121-b565-4b0e-a346-1db365cbdfa0 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:28.191768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cd121-b565-4b0e-a346-1db365cbdfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:28.199381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:28.217615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c7800-ad95-4daa-93bc-1a0881d63b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:28.218990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7800-ad95-4daa-93bc-1a0881d63b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:28.229097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7800-ad95-4daa-93bc-1a0881d63b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:28.229791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c7800-ad95-4daa-93bc-1a0881d63b45 admin admin] PATH_INFO: `/v3/projects/9c911929b7c448449f4168bcd591c4d1/users/d5633dda8c0745949bf2c106dc48b478/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:28.300382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-511cd121-b565-4b0e-a346-1db365cbdfa0 None None] MFA Rules not processed for user `e133231fd3264083a2184b9685f212d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:28.308220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-511cd121-b565-4b0e-a346-1db365cbdfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:28.345284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c7800-ad95-4daa-93bc-1a0881d63b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c911929b7c448449f4168bcd591c4d1, user_id=d5633dda8c0745949bf2c106dc48b478, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:28.355225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c7800-ad95-4daa-93bc-1a0881d63b45 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:28.555752 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1182/2312] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:05:28 2022] PUT /identity/v3/projects/9c911929b7c448449f4168bcd591c4d1/users/d5633dda8c0745949bf2c106dc48b478/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:28.579643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b40aa-1e94-4e7f-a961-24c716c07bb6 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:28.579643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b40aa-1e94-4e7f-a961-24c716c07bb6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:28.579643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b40aa-1e94-4e7f-a961-24c716c07bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:28.592144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1131/2313] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:28.636125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cd121-b565-4b0e-a346-1db365cbdfa0 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:28.638760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-511cd121-b565-4b0e-a346-1db365cbdfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:28.686125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-a75b40aa-1e94-4e7f-a961-24c716c07bb6 None None] MFA Rules not processed for user `d5633dda8c0745949bf2c106dc48b478`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:28.688332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cd121-b565-4b0e-a346-1db365cbdfa0 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:28.694212 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a75b40aa-1e94-4e7f-a961-24c716c07bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:28.870910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:28.879251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:28.879685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:28.881200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] PATH_INFO: `/v3/users/05865668aa914178ad7ea63671a05fb2` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:28.885308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05865668aa914178ad7ea63671a05fb2)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:28.891967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:28.944270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1183/2314] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:05:28.968599 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b40aa-1e94-4e7f-a961-24c716c07bb6 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:28.972243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a75b40aa-1e94-4e7f-a961-24c716c07bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:29.266497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05865668aa914178ad7ea63671a05fb2'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:29.290999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05865668aa914178ad7ea63671a05fb2'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:29.294022 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05865668aa914178ad7ea63671a05fb2'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:29.381997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b40aa-1e94-4e7f-a961-24c716c07bb6 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:29.401769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78f6c8-5d4e-46a7-beb5-dcaa54493ffe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:29.402588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78f6c8-5d4e-46a7-beb5-dcaa54493ffe service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:29.402941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78f6c8-5d4e-46a7-beb5-dcaa54493ffe service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:29.403223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78f6c8-5d4e-46a7-beb5-dcaa54493ffe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:29.406127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05865668aa914178ad7ea63671a05fb2'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:29.408960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2e78f6c8-5d4e-46a7-beb5-dcaa54493ffe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:29.430922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05865668aa914178ad7ea63671a05fb2'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:29.457343 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1132/2315] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:05:28 2022] DELETE /identity/v3/users/05865668aa914178ad7ea63671a05fb2 => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:29.460844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:29.468922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:29.491660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:29.584230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78f6c8-5d4e-46a7-beb5-dcaa54493ffe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:29.586211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78f6c8-5d4e-46a7-beb5-dcaa54493ffe service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:29.626221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1184/2316] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:05:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:05:29.633988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d22b8-0b20-44fd-a823-007d883cc263 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:29.634587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d22b8-0b20-44fd-a823-007d883cc263 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:29.635079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d22b8-0b20-44fd-a823-007d883cc263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:29.694382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc80a55-5be9-4096-8265-53b56b77c257 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:29.702934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fa2eb-9fcf-4740-8518-1029ac034660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:29.703959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fa2eb-9fcf-4740-8518-1029ac034660 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:29.704494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fa2eb-9fcf-4740-8518-1029ac034660 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:29.705336 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fa2eb-9fcf-4740-8518-1029ac034660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:29.708989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-624fa2eb-9fcf-4740-8518-1029ac034660 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:29.720970 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-ae0d22b8-0b20-44fd-a823-007d883cc263 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:29.723454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ae0d22b8-0b20-44fd-a823-007d883cc263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:29.901423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fa2eb-9fcf-4740-8518-1029ac034660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:29.903129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fa2eb-9fcf-4740-8518-1029ac034660 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:29.959927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1133/2317] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:05:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:05:29.988495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1185/2318] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:30.005950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fa2eb-9fcf-4740-8518-1029ac034660 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:30.007676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-624fa2eb-9fcf-4740-8518-1029ac034660 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:30.229311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fa2eb-9fcf-4740-8518-1029ac034660 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:30.243470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7618ee-76ad-4990-b0e5-716e74d287f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:30.244134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7618ee-76ad-4990-b0e5-716e74d287f4 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:30.244505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7618ee-76ad-4990-b0e5-716e74d287f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:30.244670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7618ee-76ad-4990-b0e5-716e74d287f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:30.274857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7618ee-76ad-4990-b0e5-716e74d287f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:30.280239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7618ee-76ad-4990-b0e5-716e74d287f4 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:30.322199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1134/2319] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:05:30.487844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:31.572204 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:32.615324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d22b8-0b20-44fd-a823-007d883cc263 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:32.621667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ae0d22b8-0b20-44fd-a823-007d883cc263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:32.751137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d22b8-0b20-44fd-a823-007d883cc263 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:32.818251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:32.828633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:32.829510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:32.830402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] PATH_INFO: `/v3/projects/de6a8daec9374e69a5d35e3ab474a0f6` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:32.845349 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de6a8daec9374e69a5d35e3ab474a0f6)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:32.849763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:33.054032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7618ee-76ad-4990-b0e5-716e74d287f4 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:33.064443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0f7618ee-76ad-4990-b0e5-716e74d287f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:33.307861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7618ee-76ad-4990-b0e5-716e74d287f4 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:33.353644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de6a8daec9374e69a5d35e3ab474a0f6', 'request_id': 'req-3d39f331-82c8-472c-a14a-58bda44f5fb7'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:33.383357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de6a8daec9374e69a5d35e3ab474a0f6', 'request_id': 'req-3d39f331-82c8-472c-a14a-58bda44f5fb7'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:33.535892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de6a8daec9374e69a5d35e3ab474a0f6', 'request_id': 'req-3d39f331-82c8-472c-a14a-58bda44f5fb7'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:33.558301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49fe6c-cecb-4302-951f-b3f0497ef54e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:33.561087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49fe6c-cecb-4302-951f-b3f0497ef54e service placement] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:33.563796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49fe6c-cecb-4302-951f-b3f0497ef54e service placement] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:33.565236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49fe6c-cecb-4302-951f-b3f0497ef54e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:33.571205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] The token cache is being invalidate because project de6a8daec9374e69a5d35e3ab474a0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:05:33.573104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:33.577520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d39f331-82c8-472c-a14a-58bda44f5fb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:05:33.582389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1c49fe6c-cecb-4302-951f-b3f0497ef54e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:33.584616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1186/2320] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:05:32 2022] DELETE /identity/v3/projects/de6a8daec9374e69a5d35e3ab474a0f6 => generated 0 bytes in 980 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:05:33.732152 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49fe6c-cecb-4302-951f-b3f0497ef54e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:33.765011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49fe6c-cecb-4302-951f-b3f0497ef54e service placement] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:33.894583 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1135/2321] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:05:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 841 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:05:34.075317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:38.895862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:48.242842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3b91-b861-446f-813d-45bed407222a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:48.244109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3b91-b861-446f-813d-45bed407222a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:48.244109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3b91-b861-446f-813d-45bed407222a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:48.290815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e5cd3b91-b861-446f-813d-45bed407222a None None] MFA Rules not processed for user `2b8dc16b63584a46b9f63b52aa5ce391`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:48.296806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e5cd3b91-b861-446f-813d-45bed407222a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:48.387751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:48.397221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1187/2322] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:51.669974 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=74 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:05:57.927089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2c23a-39d9-4ff6-aa9a-a8476b2170cc None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:57.927670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2c23a-39d9-4ff6-aa9a-a8476b2170cc None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:57.927670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2c23a-39d9-4ff6-aa9a-a8476b2170cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:57.953323 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-51e2c23a-39d9-4ff6-aa9a-a8476b2170cc None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:05:57.955602 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-51e2c23a-39d9-4ff6-aa9a-a8476b2170cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:58.062197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1136/2323] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:05:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:05:58.076198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd3b91-b861-446f-813d-45bed407222a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:05:58.077889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e5cd3b91-b861-446f-813d-45bed407222a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:05:58.166872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd3b91-b861-446f-813d-45bed407222a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:05:58.173836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9709bd-69bd-4c41-9dfe-5624010b99ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:05:58.174530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9709bd-69bd-4c41-9dfe-5624010b99ef service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:05:58.174818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9709bd-69bd-4c41-9dfe-5624010b99ef service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:05:58.175163 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9709bd-69bd-4c41-9dfe-5624010b99ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:05:58.187808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9709bd-69bd-4c41-9dfe-5624010b99ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:05:58.189507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9709bd-69bd-4c41-9dfe-5624010b99ef service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:05:58.199817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1188/2324] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:05:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 12:06:08.054590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=25 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:08.195439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=32 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:12.163522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85347a93-928c-4cdb-aa6b-434beabe378b None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:12.164194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85347a93-928c-4cdb-aa6b-434beabe378b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:12.164194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85347a93-928c-4cdb-aa6b-434beabe378b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:12.189760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-85347a93-928c-4cdb-aa6b-434beabe378b None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:12.191327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-85347a93-928c-4cdb-aa6b-434beabe378b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:12.200833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1137/2325] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:12.215908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9709bd-69bd-4c41-9dfe-5624010b99ef service swift] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:12.230400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9709bd-69bd-4c41-9dfe-5624010b99ef service swift] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:12.236770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f47388-3c98-4aa2-986e-1602cf4e850b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:12.237425 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f47388-3c98-4aa2-986e-1602cf4e850b service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:12.237731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f47388-3c98-4aa2-986e-1602cf4e850b service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:12.237984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f47388-3c98-4aa2-986e-1602cf4e850b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:12.247573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f47388-3c98-4aa2-986e-1602cf4e850b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:12.261383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f47388-3c98-4aa2-986e-1602cf4e850b service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:12.261383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1189/2326] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:06:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:06:22.179794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:22.256250 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=6 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:33.156793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eafe2e-3cbf-43f4-b35a-cfdfa5f62c74 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:33.157450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eafe2e-3cbf-43f4-b35a-cfdfa5f62c74 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:33.157755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eafe2e-3cbf-43f4-b35a-cfdfa5f62c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:33.189598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a0eafe2e-3cbf-43f4-b35a-cfdfa5f62c74 None None] MFA Rules not processed for user `76771369a0df4c5189e42291ebdf9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:33.191285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a0eafe2e-3cbf-43f4-b35a-cfdfa5f62c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:33.278570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1138/2327] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:06:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:35.788897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f47388-3c98-4aa2-986e-1602cf4e850b service swift] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:35.797140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-44f47388-3c98-4aa2-986e-1602cf4e850b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:35.920461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f47388-3c98-4aa2-986e-1602cf4e850b service swift] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:35.926596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b6dbd-e320-4cb4-902d-78c847ba2c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:35.927230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b6dbd-e320-4cb4-902d-78c847ba2c1e service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:35.927862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b6dbd-e320-4cb4-902d-78c847ba2c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:35.928149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b6dbd-e320-4cb4-902d-78c847ba2c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:35.930087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e93b6dbd-e320-4cb4-902d-78c847ba2c1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:36.131280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b6dbd-e320-4cb4-902d-78c847ba2c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:36.134311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b6dbd-e320-4cb4-902d-78c847ba2c1e service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:36.180276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1190/2328] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:06:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:06:43.271483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=25 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:44.873366 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eafe2e-3cbf-43f4-b35a-cfdfa5f62c74 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:44.876010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a0eafe2e-3cbf-43f4-b35a-cfdfa5f62c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:45.023829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eafe2e-3cbf-43f4-b35a-cfdfa5f62c74 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:45.031240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:45.032009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:45.032398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:45.032752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] PATH_INFO: `/v3/users/d5633dda8c0745949bf2c106dc48b478` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:45.034262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5633dda8c0745949bf2c106dc48b478)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:45.036722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:45.483544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5633dda8c0745949bf2c106dc48b478'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:45.507436 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5633dda8c0745949bf2c106dc48b478'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:45.508494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5633dda8c0745949bf2c106dc48b478'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:45.599628 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5633dda8c0745949bf2c106dc48b478'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:45.648808 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5633dda8c0745949bf2c106dc48b478'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:45.662344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1139/2329] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:06:44 2022] DELETE /identity/v3/users/d5633dda8c0745949bf2c106dc48b478 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:06:45.667484 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99629b2f-c2d9-4b83-9770-d25ffd2e5250 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:45.668576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99629b2f-c2d9-4b83-9770-d25ffd2e5250 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:45.668978 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99629b2f-c2d9-4b83-9770-d25ffd2e5250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:45.731205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-99629b2f-c2d9-4b83-9770-d25ffd2e5250 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:45.738900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-99629b2f-c2d9-4b83-9770-d25ffd2e5250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:45.864371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1191/2330] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:45.880985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:45.884341 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:45.983104 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd482fe-8de8-4e43-939b-5c952238a07c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:45.988639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58cc73-25c9-4ce4-af08-fce9c2dee392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:45.989490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58cc73-25c9-4ce4-af08-fce9c2dee392 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:45.989909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58cc73-25c9-4ce4-af08-fce9c2dee392 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:45.990370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58cc73-25c9-4ce4-af08-fce9c2dee392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:46.003896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58cc73-25c9-4ce4-af08-fce9c2dee392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:46.006187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58cc73-25c9-4ce4-af08-fce9c2dee392 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:46.018318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1140/2331] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:06:47.345733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99629b2f-c2d9-4b83-9770-d25ffd2e5250 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:47.347213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-99629b2f-c2d9-4b83-9770-d25ffd2e5250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:47.384359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99629b2f-c2d9-4b83-9770-d25ffd2e5250 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:47.415823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:47.416793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:47.417193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:47.417586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] PATH_INFO: `/v3/projects/9c911929b7c448449f4168bcd591c4d1` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:47.420234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c911929b7c448449f4168bcd591c4d1)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:47.422314 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:47.466162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e226b6-ef98-41e4-8f0e-b1f05ea3b9e9 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:47.467062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e226b6-ef98-41e4-8f0e-b1f05ea3b9e9 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:47.467520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e226b6-ef98-41e4-8f0e-b1f05ea3b9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:47.494585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f0e226b6-ef98-41e4-8f0e-b1f05ea3b9e9 None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:47.495853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f0e226b6-ef98-41e4-8f0e-b1f05ea3b9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:47.522318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:47.620954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1141/2332] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 4 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:06:47.638870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:47.639926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e226b6-ef98-41e4-8f0e-b1f05ea3b9e9 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:47.642627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f0e226b6-ef98-41e4-8f0e-b1f05ea3b9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:47.665654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c911929b7c448449f4168bcd591c4d1', 'request_id': 'req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:47.689445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c911929b7c448449f4168bcd591c4d1', 'request_id': 'req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:47.792730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e226b6-ef98-41e4-8f0e-b1f05ea3b9e9 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:47.836665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c911929b7c448449f4168bcd591c4d1', 'request_id': 'req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:47.856905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] The token cache is being invalidate because project 9c911929b7c448449f4168bcd591c4d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:06:47.857606 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:47.858727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1b137cf7-5d4c-4aea-95d5-37ccaa89e2df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:47.860836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1192/2333] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:06:47 2022] DELETE /identity/v3/projects/9c911929b7c448449f4168bcd591c4d1 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:06:47.863265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c23ac-af8a-402f-b351-1f3eda6c8cfe None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:47.863914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c23ac-af8a-402f-b351-1f3eda6c8cfe None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:47.864924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c23ac-af8a-402f-b351-1f3eda6c8cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:47.883446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feae824-9e7b-43b0-9e6e-b7a63343e2eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:47.884193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feae824-9e7b-43b0-9e6e-b7a63343e2eb service swift] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:47.884559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feae824-9e7b-43b0-9e6e-b7a63343e2eb service swift] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:47.884904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feae824-9e7b-43b0-9e6e-b7a63343e2eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:47.887799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8feae824-9e7b-43b0-9e6e-b7a63343e2eb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:47.895196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-848c23ac-af8a-402f-b351-1f3eda6c8cfe None None] MFA Rules not processed for user `f3f607e48e534dd09c3619982978e9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:47.896875 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-848c23ac-af8a-402f-b351-1f3eda6c8cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:47.968978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feae824-9e7b-43b0-9e6e-b7a63343e2eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:47.971107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feae824-9e7b-43b0-9e6e-b7a63343e2eb service swift] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:47.978242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1193/2334] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:47.985335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f704b0-90d8-4753-9c4a-833420d912da None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:47.985619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f704b0-90d8-4753-9c4a-833420d912da None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:47.986071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f704b0-90d8-4753-9c4a-833420d912da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:48.004443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1142/2335] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:06:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 12:06:48.019855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-33f704b0-90d8-4753-9c4a-833420d912da None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:48.021791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-33f704b0-90d8-4753-9c4a-833420d912da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:48.100265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1194/2336] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:48.110052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4918159-e995-44d2-b47a-71cd4bd7f404 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:48.110753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4918159-e995-44d2-b47a-71cd4bd7f404 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:48.110929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4918159-e995-44d2-b47a-71cd4bd7f404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:48.134185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f4918159-e995-44d2-b47a-71cd4bd7f404 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:48.135452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f4918159-e995-44d2-b47a-71cd4bd7f404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:48.144330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1143/2337] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:48.149980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f704b0-90d8-4753-9c4a-833420d912da None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:48.161931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f704b0-90d8-4753-9c4a-833420d912da None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:48.171064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f82d62-f0e9-4295-829b-6ce589d224bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:48.172593 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f82d62-f0e9-4295-829b-6ce589d224bb admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:48.173217 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f82d62-f0e9-4295-829b-6ce589d224bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:48.173973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f82d62-f0e9-4295-829b-6ce589d224bb admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:48.175248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f82d62-f0e9-4295-829b-6ce589d224bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:06:48.175881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f82d62-f0e9-4295-829b-6ce589d224bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:48.179109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f82d62-f0e9-4295-829b-6ce589d224bb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:48.204726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1195/2338] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:06:48.210468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4918159-e995-44d2-b47a-71cd4bd7f404 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:48.230076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4918159-e995-44d2-b47a-71cd4bd7f404 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:48.238878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ff649-b992-47ce-b927-348e2f9b15f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:48.240028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ff649-b992-47ce-b927-348e2f9b15f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:48.240715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ff649-b992-47ce-b927-348e2f9b15f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:48.241188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ff649-b992-47ce-b927-348e2f9b15f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:48.242701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ff649-b992-47ce-b927-348e2f9b15f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:48.247838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ff649-b992-47ce-b927-348e2f9b15f0 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:48.431264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1144/2339] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:06:48 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:06:48.436431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f82d62-f0e9-4295-829b-6ce589d224bb admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:48.455401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f82d62-f0e9-4295-829b-6ce589d224bb admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:48.567313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98f8fa-c90f-44d6-b169-805ce008edaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:48.568445 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98f8fa-c90f-44d6-b169-805ce008edaa admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:48.568950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98f8fa-c90f-44d6-b169-805ce008edaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:48.569679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98f8fa-c90f-44d6-b169-805ce008edaa admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:48.572206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98f8fa-c90f-44d6-b169-805ce008edaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:48.576974 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98f8fa-c90f-44d6-b169-805ce008edaa admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:48.745678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1196/2340] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:06:48 2022] POST /identity/v3/users => generated 287 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:06:48.752071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ff649-b992-47ce-b927-348e2f9b15f0 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:48.767689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ff649-b992-47ce-b927-348e2f9b15f0 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:48.781884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4f50e-a9e1-49e3-a320-f49ee472b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:48.781884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4f50e-a9e1-49e3-a320-f49ee472b285 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:48.781884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4f50e-a9e1-49e3-a320-f49ee472b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:48.781884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4f50e-a9e1-49e3-a320-f49ee472b285 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:48.782491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4f50e-a9e1-49e3-a320-f49ee472b285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:48.782491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4f50e-a9e1-49e3-a320-f49ee472b285 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:48.800416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1145/2341] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:06:48.804822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98f8fa-c90f-44d6-b169-805ce008edaa admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:48.824778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98f8fa-c90f-44d6-b169-805ce008edaa admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:48.833047 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2747b-87ca-4211-8c4a-4b4b1a0b2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:48.834035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2747b-87ca-4211-8c4a-4b4b1a0b2d54 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:48.834460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2747b-87ca-4211-8c4a-4b4b1a0b2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:48.834855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2747b-87ca-4211-8c4a-4b4b1a0b2d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:48.835883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2747b-87ca-4211-8c4a-4b4b1a0b2d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:48.839192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2747b-87ca-4211-8c4a-4b4b1a0b2d54 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:48.861599 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1197/2342] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:06:48.865645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4f50e-a9e1-49e3-a320-f49ee472b285 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:48.877801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4f50e-a9e1-49e3-a320-f49ee472b285 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:48.884233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384da81-1ae4-426f-ac20-4374367632ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:48.888538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384da81-1ae4-426f-ac20-4374367632ca admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:48.888538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384da81-1ae4-426f-ac20-4374367632ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:48.888538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384da81-1ae4-426f-ac20-4374367632ca admin admin] PATH_INFO: `/v3/projects/103dd7a193974ae295b939a8a1b44e67/users/7e39d486794546b8b895f5979905d91c/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:48.906946 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384da81-1ae4-426f-ac20-4374367632ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=103dd7a193974ae295b939a8a1b44e67, user_id=7e39d486794546b8b895f5979905d91c, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:48.912073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384da81-1ae4-426f-ac20-4374367632ca admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:49.214825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1146/2343] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:06:48 2022] PUT /identity/v3/projects/103dd7a193974ae295b939a8a1b44e67/users/7e39d486794546b8b895f5979905d91c/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:06:49.220559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc7efa-e03b-4e9d-b0c5-10b254d1ec3c None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:49.220829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc7efa-e03b-4e9d-b0c5-10b254d1ec3c None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:49.221114 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc7efa-e03b-4e9d-b0c5-10b254d1ec3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:49.253653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-a9bc7efa-e03b-4e9d-b0c5-10b254d1ec3c None None] MFA Rules not processed for user `7e39d486794546b8b895f5979905d91c`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:49.254922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a9bc7efa-e03b-4e9d-b0c5-10b254d1ec3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:49.329576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1198/2344] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:49.347231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:49.356600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384da81-1ae4-426f-ac20-4374367632ca admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:49.360029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c384da81-1ae4-426f-ac20-4374367632ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:49.465280 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384da81-1ae4-426f-ac20-4374367632ca admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:49.470135 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3a748-035a-4ccf-8069-6b3ebda6f8ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:49.470758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3a748-035a-4ccf-8069-6b3ebda6f8ac service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:49.471333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3a748-035a-4ccf-8069-6b3ebda6f8ac service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:49.471703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3a748-035a-4ccf-8069-6b3ebda6f8ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:49.485848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3a748-035a-4ccf-8069-6b3ebda6f8ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:49.489811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3a748-035a-4ccf-8069-6b3ebda6f8ac service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:49.519896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1147/2345] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:06:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:06:49.560730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc7efa-e03b-4e9d-b0c5-10b254d1ec3c None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:49.653380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc7efa-e03b-4e9d-b0c5-10b254d1ec3c None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:49.661830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:49.663319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:49.663716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:49.664460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] PATH_INFO: `/v3/users/7e39d486794546b8b895f5979905d91c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:49.667249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e39d486794546b8b895f5979905d91c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:49.669799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:49.817403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:49.956357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e39d486794546b8b895f5979905d91c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:49.957058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e39d486794546b8b895f5979905d91c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:49.965152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e39d486794546b8b895f5979905d91c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:49.966106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e39d486794546b8b895f5979905d91c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:50.077889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e39d486794546b8b895f5979905d91c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:50.098727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1199/2346] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:06:49 2022] DELETE /identity/v3/users/7e39d486794546b8b895f5979905d91c => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:06:50.104975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25952d-1554-41a5-93b9-46f2c0f6470f None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:50.105381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25952d-1554-41a5-93b9-46f2c0f6470f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:50.105722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25952d-1554-41a5-93b9-46f2c0f6470f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:50.132419 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-ac25952d-1554-41a5-93b9-46f2c0f6470f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:50.134183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ac25952d-1554-41a5-93b9-46f2c0f6470f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:50.214162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1148/2347] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:50.227792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:50.229422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:50.314949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ba5c5-84c8-45e5-af48-4a0de08c00d7 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:50.319425 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c5165-a5af-4f6f-bc1d-fd1a175b76a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:50.320251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c5165-a5af-4f6f-bc1d-fd1a175b76a0 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:50.320799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c5165-a5af-4f6f-bc1d-fd1a175b76a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:50.321145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c5165-a5af-4f6f-bc1d-fd1a175b76a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:50.333112 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c5165-a5af-4f6f-bc1d-fd1a175b76a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:50.334765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c5165-a5af-4f6f-bc1d-fd1a175b76a0 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:50.346888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1200/2348] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:06:51.010202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25952d-1554-41a5-93b9-46f2c0f6470f None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:51.012331 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ac25952d-1554-41a5-93b9-46f2c0f6470f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:51.028420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25952d-1554-41a5-93b9-46f2c0f6470f None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:51.044821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:51.045868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:51.046441 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:51.046884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] PATH_INFO: `/v3/projects/103dd7a193974ae295b939a8a1b44e67` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:51.048661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=103dd7a193974ae295b939a8a1b44e67)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:51.051505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:51.240720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:51.244738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '103dd7a193974ae295b939a8a1b44e67', 'request_id': 'req-25856f84-a5c6-469a-aadf-d5fa9d144f36'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:51.252241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '103dd7a193974ae295b939a8a1b44e67', 'request_id': 'req-25856f84-a5c6-469a-aadf-d5fa9d144f36'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:51.261302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '103dd7a193974ae295b939a8a1b44e67', 'request_id': 'req-25856f84-a5c6-469a-aadf-d5fa9d144f36'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:51.422845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] The token cache is being invalidate because project 103dd7a193974ae295b939a8a1b44e67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:06:51.423757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:51.425311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:51.426106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-25856f84-a5c6-469a-aadf-d5fa9d144f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:51.431284 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1149/2349] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:06:51 2022] DELETE /identity/v3/projects/103dd7a193974ae295b939a8a1b44e67 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:06:52.811275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0547a-0175-48fc-9c87-a8c30ddb531e None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:52.813103 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0547a-0175-48fc-9c87-a8c30ddb531e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:52.813668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0547a-0175-48fc-9c87-a8c30ddb531e None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:52.818051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1201/2350] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:06:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:06:52.845398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a08d8-52e8-448d-99ac-90f3eb64af63 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:52.846897 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a08d8-52e8-448d-99ac-90f3eb64af63 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:52.847644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a08d8-52e8-448d-99ac-90f3eb64af63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:52.886987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-463a08d8-52e8-448d-99ac-90f3eb64af63 None None] MFA Rules not processed for user `16374dbb6d084ddfa3b49afd19c7c325`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:52.886987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-463a08d8-52e8-448d-99ac-90f3eb64af63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:53.014835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1150/2351] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:06:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:53.027164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0547a-0175-48fc-9c87-a8c30ddb531e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:53.028997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6ba0547a-0175-48fc-9c87-a8c30ddb531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:53.061920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0547a-0175-48fc-9c87-a8c30ddb531e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:53.069392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a46c27-2abe-4ca1-aa96-a0f4770566ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:53.070114 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a46c27-2abe-4ca1-aa96-a0f4770566ce service placement] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:53.071098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a46c27-2abe-4ca1-aa96-a0f4770566ce service placement] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:53.071714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a46c27-2abe-4ca1-aa96-a0f4770566ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:53.074606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a08d8-52e8-448d-99ac-90f3eb64af63 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:53.077589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-463a08d8-52e8-448d-99ac-90f3eb64af63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:53.088119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a46c27-2abe-4ca1-aa96-a0f4770566ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:53.090826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a46c27-2abe-4ca1-aa96-a0f4770566ce service placement] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:53.108963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1202/2352] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:06:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:06:53.191054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a08d8-52e8-448d-99ac-90f3eb64af63 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:53.199193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:53.200230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:53.200729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:53.201224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] PATH_INFO: `/v3/users/ee6a9052a75a4d8c9f4a08638685012b` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:53.203244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee6a9052a75a4d8c9f4a08638685012b)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:53.208901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:53.522519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee6a9052a75a4d8c9f4a08638685012b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:53.536023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee6a9052a75a4d8c9f4a08638685012b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:53.536369 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee6a9052a75a4d8c9f4a08638685012b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:53.644037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee6a9052a75a4d8c9f4a08638685012b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:53.677745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:53.691844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6562a8b4-5f52-44be-a1e0-42f676ff894d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee6a9052a75a4d8c9f4a08638685012b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:53.705638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1151/2353] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:06:53 2022] DELETE /identity/v3/users/ee6a9052a75a4d8c9f4a08638685012b => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:06:54.137737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a46c27-2abe-4ca1-aa96-a0f4770566ce service placement] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:54.176480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a46c27-2abe-4ca1-aa96-a0f4770566ce service placement] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:54.269147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:54.269920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:54.270125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:54.270455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] PATH_INFO: `/v3/projects/df93e841cba9441f84c15a826db25de6` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:54.271869 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=df93e841cba9441f84c15a826db25de6)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:54.274530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:54.416417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df93e841cba9441f84c15a826db25de6', 'request_id': 'req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:54.423593 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df93e841cba9441f84c15a826db25de6', 'request_id': 'req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:54.521756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df93e841cba9441f84c15a826db25de6', 'request_id': 'req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:54.536513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] The token cache is being invalidate because project df93e841cba9441f84c15a826db25de6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:06:54.536809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:54.537511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ea6eac23-894c-404f-83cd-d4d6d3a2d2fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:06:54.540006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1203/2354] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:06:54 2022] DELETE /identity/v3/projects/df93e841cba9441f84c15a826db25de6 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:06:54.545002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92ae94-2f57-447f-b4e4-0f5f80696de7 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:54.545806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92ae94-2f57-447f-b4e4-0f5f80696de7 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:54.545806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92ae94-2f57-447f-b4e4-0f5f80696de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:54.567917 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-ff92ae94-2f57-447f-b4e4-0f5f80696de7 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:54.569574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ff92ae94-2f57-447f-b4e4-0f5f80696de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:54.643665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1152/2355] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:54.651986 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7f2e2-be07-461a-ac35-c8d81fb81b09 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:54.652725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7f2e2-be07-461a-ac35-c8d81fb81b09 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:54.653957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7f2e2-be07-461a-ac35-c8d81fb81b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:54.678427 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2fe7f2e2-be07-461a-ac35-c8d81fb81b09 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:54.679705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2fe7f2e2-be07-461a-ac35-c8d81fb81b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:54.689044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1204/2356] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:54.693745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92ae94-2f57-447f-b4e4-0f5f80696de7 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:54.705792 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92ae94-2f57-447f-b4e4-0f5f80696de7 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:54.711451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e7716-30bb-4b9f-9d57-d2f19eb4f1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:54.712364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e7716-30bb-4b9f-9d57-d2f19eb4f1af admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:54.712649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e7716-30bb-4b9f-9d57-d2f19eb4f1af admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:54.713079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e7716-30bb-4b9f-9d57-d2f19eb4f1af admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:54.713766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e7716-30bb-4b9f-9d57-d2f19eb4f1af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:06:54.714071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e7716-30bb-4b9f-9d57-d2f19eb4f1af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:54.715817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e7716-30bb-4b9f-9d57-d2f19eb4f1af admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:54.730622 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1153/2357] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:06:54.735476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7f2e2-be07-461a-ac35-c8d81fb81b09 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:54.752556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7f2e2-be07-461a-ac35-c8d81fb81b09 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:54.761686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f12370-da13-4566-a13c-883ae85d6428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:54.762842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f12370-da13-4566-a13c-883ae85d6428 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:54.763415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f12370-da13-4566-a13c-883ae85d6428 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:54.763955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f12370-da13-4566-a13c-883ae85d6428 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:54.765315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f12370-da13-4566-a13c-883ae85d6428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:54.769071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f12370-da13-4566-a13c-883ae85d6428 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:54.778087 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:54.951120 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1205/2358] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:06:54 2022] POST /identity/v3/projects => generated 373 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:06:54.952861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e7716-30bb-4b9f-9d57-d2f19eb4f1af admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:54.967658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e7716-30bb-4b9f-9d57-d2f19eb4f1af admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:55.036538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b5117-e160-4f5d-82d1-5354e462ba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:55.037073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5117-e160-4f5d-82d1-5354e462ba36 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:55.037355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5117-e160-4f5d-82d1-5354e462ba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:55.037635 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b5117-e160-4f5d-82d1-5354e462ba36 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:55.038276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b5117-e160-4f5d-82d1-5354e462ba36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:55.040905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b5117-e160-4f5d-82d1-5354e462ba36 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:55.222451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1154/2359] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:06:54 2022] POST /identity/v3/users => generated 287 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:06:55.228057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f12370-da13-4566-a13c-883ae85d6428 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:55.250543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f12370-da13-4566-a13c-883ae85d6428 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:55.260391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fca79e-9f9a-4d02-ac68-248a6d44e99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:55.261764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fca79e-9f9a-4d02-ac68-248a6d44e99d admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:55.262619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fca79e-9f9a-4d02-ac68-248a6d44e99d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:55.263260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fca79e-9f9a-4d02-ac68-248a6d44e99d admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:55.264657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fca79e-9f9a-4d02-ac68-248a6d44e99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:55.268881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fca79e-9f9a-4d02-ac68-248a6d44e99d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:55.290331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1206/2360] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:06:55.297150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b5117-e160-4f5d-82d1-5354e462ba36 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:55.318029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b5117-e160-4f5d-82d1-5354e462ba36 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:55.329610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8827cf-3ee6-43f5-a24b-a285a1c2667a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:55.332696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8827cf-3ee6-43f5-a24b-a285a1c2667a admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:55.333326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8827cf-3ee6-43f5-a24b-a285a1c2667a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:55.334009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8827cf-3ee6-43f5-a24b-a285a1c2667a admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:55.335401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8827cf-3ee6-43f5-a24b-a285a1c2667a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:55.338547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8827cf-3ee6-43f5-a24b-a285a1c2667a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:55.356520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1155/2361] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:06:55.362525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fca79e-9f9a-4d02-ac68-248a6d44e99d admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:55.380540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fca79e-9f9a-4d02-ac68-248a6d44e99d admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:55.388621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97714db6-8ca0-47cc-b202-2783de731caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:55.389576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97714db6-8ca0-47cc-b202-2783de731caf admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:55.389944 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97714db6-8ca0-47cc-b202-2783de731caf admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:55.390202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97714db6-8ca0-47cc-b202-2783de731caf admin admin] PATH_INFO: `/v3/projects/52da6fc98c474b1fb9bb78cf2c7f1bf3/users/0ad02104559f4eec9182c50ab13b7c48/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:55.410805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97714db6-8ca0-47cc-b202-2783de731caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=52da6fc98c474b1fb9bb78cf2c7f1bf3, user_id=0ad02104559f4eec9182c50ab13b7c48, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:55.413764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97714db6-8ca0-47cc-b202-2783de731caf admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:55.490876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1207/2362] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:06:55 2022] PUT /identity/v3/projects/52da6fc98c474b1fb9bb78cf2c7f1bf3/users/0ad02104559f4eec9182c50ab13b7c48/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:06:55.504421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bded2a7-c08a-470e-9987-1c3333cfff69 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:55.504929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bded2a7-c08a-470e-9987-1c3333cfff69 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:55.505831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bded2a7-c08a-470e-9987-1c3333cfff69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:55.561305 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-5bded2a7-c08a-470e-9987-1c3333cfff69 None None] MFA Rules not processed for user `0ad02104559f4eec9182c50ab13b7c48`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:06:55.563500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5bded2a7-c08a-470e-9987-1c3333cfff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:55.645401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:06:55.690391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1156/2363] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:06:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:06:55.709124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97714db6-8ca0-47cc-b202-2783de731caf admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:06:55.711454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-97714db6-8ca0-47cc-b202-2783de731caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:06:55.845683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97714db6-8ca0-47cc-b202-2783de731caf admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:06:55.855095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939bae1-e0e5-43aa-a33e-20c16c64cc55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:06:55.856326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939bae1-e0e5-43aa-a33e-20c16c64cc55 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:06:55.856921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939bae1-e0e5-43aa-a33e-20c16c64cc55 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:06:55.857227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939bae1-e0e5-43aa-a33e-20c16c64cc55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:06:55.873551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939bae1-e0e5-43aa-a33e-20c16c64cc55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:06:55.876871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939bae1-e0e5-43aa-a33e-20c16c64cc55 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:06:55.894497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1208/2364] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:06:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:07:05.440624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bded2a7-c08a-470e-9987-1c3333cfff69 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:05.445862 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5bded2a7-c08a-470e-9987-1c3333cfff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:05.515652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bded2a7-c08a-470e-9987-1c3333cfff69 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:05.522198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc4129-68a2-4548-a2e4-26e7f6edc116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:05.523686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc4129-68a2-4548-a2e4-26e7f6edc116 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:05.524198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc4129-68a2-4548-a2e4-26e7f6edc116 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:05.524660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc4129-68a2-4548-a2e4-26e7f6edc116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:05.527561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5bcc4129-68a2-4548-a2e4-26e7f6edc116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:05.657937 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc4129-68a2-4548-a2e4-26e7f6edc116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:05.661379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc4129-68a2-4548-a2e4-26e7f6edc116 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:05.671492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1157/2365] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:07:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:07:05.889655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:07:09.797743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939bae1-e0e5-43aa-a33e-20c16c64cc55 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:09.936240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939bae1-e0e5-43aa-a33e-20c16c64cc55 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:09.944521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:09.946200 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:09.946674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:09.947164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] PATH_INFO: `/v3/users/0ad02104559f4eec9182c50ab13b7c48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:09.949574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad02104559f4eec9182c50ab13b7c48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:09.953527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:10.447662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad02104559f4eec9182c50ab13b7c48'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:07:10.448888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad02104559f4eec9182c50ab13b7c48'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:07:10.464580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad02104559f4eec9182c50ab13b7c48'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:07:10.465715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad02104559f4eec9182c50ab13b7c48'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:07:10.574744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad02104559f4eec9182c50ab13b7c48'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:07:10.600246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1209/2366] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:07:09 2022] DELETE /identity/v3/users/0ad02104559f4eec9182c50ab13b7c48 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:07:10.606734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c52f9b-6c98-4027-8fdd-b7fb58121d13 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:10.607365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c52f9b-6c98-4027-8fdd-b7fb58121d13 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:10.607720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c52f9b-6c98-4027-8fdd-b7fb58121d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:10.642818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-61c52f9b-6c98-4027-8fdd-b7fb58121d13 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:07:10.645023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-61c52f9b-6c98-4027-8fdd-b7fb58121d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:10.734123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1158/2367] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:07:10.745761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:10.747517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:10.832074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36c0a5-3670-4c88-a650-6499ea9ccbe5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:10.839011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecccdf2-e745-4204-87ec-76053075127c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:10.840011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecccdf2-e745-4204-87ec-76053075127c service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:10.840511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecccdf2-e745-4204-87ec-76053075127c service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:10.840986 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecccdf2-e745-4204-87ec-76053075127c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:10.857895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecccdf2-e745-4204-87ec-76053075127c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:10.860723 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecccdf2-e745-4204-87ec-76053075127c service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:10.879252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1210/2368] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:07:11.747899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c52f9b-6c98-4027-8fdd-b7fb58121d13 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:11.749471 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-61c52f9b-6c98-4027-8fdd-b7fb58121d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:11.764241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c52f9b-6c98-4027-8fdd-b7fb58121d13 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:11.772019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:11.773450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:11.773978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:11.774481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] PATH_INFO: `/v3/projects/52da6fc98c474b1fb9bb78cf2c7f1bf3` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:11.776096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52da6fc98c474b1fb9bb78cf2c7f1bf3)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:11.778485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:11.945720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:07:11.961795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52da6fc98c474b1fb9bb78cf2c7f1bf3', 'request_id': 'req-6749eb54-42a6-41af-8752-2f022dd3a1e4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:07:11.968446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52da6fc98c474b1fb9bb78cf2c7f1bf3', 'request_id': 'req-6749eb54-42a6-41af-8752-2f022dd3a1e4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:07:11.976731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52da6fc98c474b1fb9bb78cf2c7f1bf3', 'request_id': 'req-6749eb54-42a6-41af-8752-2f022dd3a1e4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:07:12.068647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] The token cache is being invalidate because project 52da6fc98c474b1fb9bb78cf2c7f1bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:07:12.069136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:07:12.070760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6749eb54-42a6-41af-8752-2f022dd3a1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:07:12.073113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1159/2369] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:07:11 2022] DELETE /identity/v3/projects/52da6fc98c474b1fb9bb78cf2c7f1bf3 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:07:12.078499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20511b-bd28-4bf5-9531-615624c623be None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:12.086122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20511b-bd28-4bf5-9531-615624c623be None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:12.086122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20511b-bd28-4bf5-9531-615624c623be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:12.101412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2c20511b-bd28-4bf5-9531-615624c623be None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:07:12.102759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2c20511b-bd28-4bf5-9531-615624c623be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:12.137751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:07:12.214214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1211/2370] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:07:12.223735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7988d8-9f12-46e4-a5b3-04eaa8c736a2 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:12.224192 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7988d8-9f12-46e4-a5b3-04eaa8c736a2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:12.225170 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7988d8-9f12-46e4-a5b3-04eaa8c736a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:12.253358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-5a7988d8-9f12-46e4-a5b3-04eaa8c736a2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:07:12.255596 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5a7988d8-9f12-46e4-a5b3-04eaa8c736a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:12.270693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1160/2371] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:07:12.276355 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20511b-bd28-4bf5-9531-615624c623be None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:12.295208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20511b-bd28-4bf5-9531-615624c623be None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:12.304468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a77ef-cabe-4a8c-9dd5-31539a6826f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:12.305461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a77ef-cabe-4a8c-9dd5-31539a6826f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:12.305948 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a77ef-cabe-4a8c-9dd5-31539a6826f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:12.306414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a77ef-cabe-4a8c-9dd5-31539a6826f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:12.307603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a77ef-cabe-4a8c-9dd5-31539a6826f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:07:12.308140 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a77ef-cabe-4a8c-9dd5-31539a6826f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:12.311395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a77ef-cabe-4a8c-9dd5-31539a6826f5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:12.336738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1212/2372] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:07:12.342503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7988d8-9f12-46e4-a5b3-04eaa8c736a2 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:12.359735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7988d8-9f12-46e4-a5b3-04eaa8c736a2 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:12.367938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986800c7-7bdd-46e2-ba40-dc3ee0ca0654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:12.369068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986800c7-7bdd-46e2-ba40-dc3ee0ca0654 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:12.369577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986800c7-7bdd-46e2-ba40-dc3ee0ca0654 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:12.370081 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986800c7-7bdd-46e2-ba40-dc3ee0ca0654 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:12.371321 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986800c7-7bdd-46e2-ba40-dc3ee0ca0654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:12.375340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986800c7-7bdd-46e2-ba40-dc3ee0ca0654 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:12.508578 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1161/2373] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:07:12 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:07:12.510773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a77ef-cabe-4a8c-9dd5-31539a6826f5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:12.528207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a77ef-cabe-4a8c-9dd5-31539a6826f5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:12.613335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c34877-ce73-4b5c-b465-2bc807847e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:12.613953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c34877-ce73-4b5c-b465-2bc807847e93 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:12.614230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c34877-ce73-4b5c-b465-2bc807847e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:12.614503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c34877-ce73-4b5c-b465-2bc807847e93 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:12.615151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c34877-ce73-4b5c-b465-2bc807847e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:12.617280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c34877-ce73-4b5c-b465-2bc807847e93 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:12.709384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1213/2374] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:07:12 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:07:12.714743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986800c7-7bdd-46e2-ba40-dc3ee0ca0654 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:12.727130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986800c7-7bdd-46e2-ba40-dc3ee0ca0654 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:12.733895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539842ae-dc15-4488-9342-5e14413dc425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:12.742019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539842ae-dc15-4488-9342-5e14413dc425 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:12.742019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539842ae-dc15-4488-9342-5e14413dc425 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:12.742019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539842ae-dc15-4488-9342-5e14413dc425 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:12.742019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539842ae-dc15-4488-9342-5e14413dc425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:12.742019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539842ae-dc15-4488-9342-5e14413dc425 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:12.752701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1162/2375] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:07:12.758370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c34877-ce73-4b5c-b465-2bc807847e93 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:12.771472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c34877-ce73-4b5c-b465-2bc807847e93 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:12.777744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c33ec0-7ba2-4028-91f2-ae8c11c8c1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:12.778762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c33ec0-7ba2-4028-91f2-ae8c11c8c1cd admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:12.788303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c33ec0-7ba2-4028-91f2-ae8c11c8c1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:12.788303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c33ec0-7ba2-4028-91f2-ae8c11c8c1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:12.788303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c33ec0-7ba2-4028-91f2-ae8c11c8c1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:12.788303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c33ec0-7ba2-4028-91f2-ae8c11c8c1cd admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:12.809207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1214/2376] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:07:12.813853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539842ae-dc15-4488-9342-5e14413dc425 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:12.840471 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539842ae-dc15-4488-9342-5e14413dc425 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:12.849927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c81d4-c52a-4918-a9e9-757ce2831395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:12.851255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c81d4-c52a-4918-a9e9-757ce2831395 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:12.851752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c81d4-c52a-4918-a9e9-757ce2831395 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:12.852290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c81d4-c52a-4918-a9e9-757ce2831395 admin admin] PATH_INFO: `/v3/projects/037b4d6af496475da87a47a9fd8d438d/users/c7340e91e66f41559bb41e620ad3229f/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:12.884636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c81d4-c52a-4918-a9e9-757ce2831395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=037b4d6af496475da87a47a9fd8d438d, user_id=c7340e91e66f41559bb41e620ad3229f, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:12.890213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c81d4-c52a-4918-a9e9-757ce2831395 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:12.997130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1163/2377] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:07:12 2022] PUT /identity/v3/projects/037b4d6af496475da87a47a9fd8d438d/users/c7340e91e66f41559bb41e620ad3229f/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:07:13.006296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ee9bb-4a3a-4a45-ac2c-6d8cab87da5a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:13.006810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ee9bb-4a3a-4a45-ac2c-6d8cab87da5a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:13.007956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ee9bb-4a3a-4a45-ac2c-6d8cab87da5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:13.041684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-594ee9bb-4a3a-4a45-ac2c-6d8cab87da5a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:07:13.043847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-594ee9bb-4a3a-4a45-ac2c-6d8cab87da5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:13.185545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1215/2378] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:07:13.203614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c81d4-c52a-4918-a9e9-757ce2831395 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:13.207670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5b9c81d4-c52a-4918-a9e9-757ce2831395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:13.302886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c81d4-c52a-4918-a9e9-757ce2831395 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:13.310038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0132b6-07e8-44fa-86b4-782ec930c9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:13.313547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0132b6-07e8-44fa-86b4-782ec930c9fd service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:13.313547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0132b6-07e8-44fa-86b4-782ec930c9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:13.313547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0132b6-07e8-44fa-86b4-782ec930c9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:13.323935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0132b6-07e8-44fa-86b4-782ec930c9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:13.326487 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0132b6-07e8-44fa-86b4-782ec930c9fd service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:13.346416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1164/2379] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:07:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:07:21.254039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947957be-1198-45a9-8700-0f6663cfb8bf None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:21.254908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947957be-1198-45a9-8700-0f6663cfb8bf None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:21.255142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947957be-1198-45a9-8700-0f6663cfb8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:21.306328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-947957be-1198-45a9-8700-0f6663cfb8bf None None] MFA Rules not processed for user `c7340e91e66f41559bb41e620ad3229f`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:07:21.308402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-947957be-1198-45a9-8700-0f6663cfb8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:21.342876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:07:21.417515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1216/2380] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:07:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:07:21.419704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0132b6-07e8-44fa-86b4-782ec930c9fd service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:21.435733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0132b6-07e8-44fa-86b4-782ec930c9fd service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:21.442449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a496c83-7fe5-4e44-b30a-145708ab2d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:21.443465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a496c83-7fe5-4e44-b30a-145708ab2d22 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:21.444474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a496c83-7fe5-4e44-b30a-145708ab2d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:21.444888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a496c83-7fe5-4e44-b30a-145708ab2d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:21.446032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a496c83-7fe5-4e44-b30a-145708ab2d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:21.448685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a496c83-7fe5-4e44-b30a-145708ab2d22 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:21.607781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1165/2381] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:07:21 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:07:21.613705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947957be-1198-45a9-8700-0f6663cfb8bf None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:21.640370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947957be-1198-45a9-8700-0f6663cfb8bf None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:21.820284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4d7e2-e98f-44d1-8d87-40b2c49237c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:21.821300 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4d7e2-e98f-44d1-8d87-40b2c49237c2 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:21.821797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4d7e2-e98f-44d1-8d87-40b2c49237c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:21.822343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4d7e2-e98f-44d1-8d87-40b2c49237c2 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:21.823438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4d7e2-e98f-44d1-8d87-40b2c49237c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:21.827350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4d7e2-e98f-44d1-8d87-40b2c49237c2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:21.986870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1217/2382] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:07:21 2022] POST /identity/v3/users => generated 293 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:07:21.991618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a496c83-7fe5-4e44-b30a-145708ab2d22 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:22.006330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a496c83-7fe5-4e44-b30a-145708ab2d22 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:22.012404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c9e4c-4781-470e-9d4a-b5abe8918659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:22.013358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c9e4c-4781-470e-9d4a-b5abe8918659 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:22.013625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c9e4c-4781-470e-9d4a-b5abe8918659 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:22.013909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c9e4c-4781-470e-9d4a-b5abe8918659 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:22.014598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c9e4c-4781-470e-9d4a-b5abe8918659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:22.016352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c9e4c-4781-470e-9d4a-b5abe8918659 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:22.030013 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1166/2383] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:07:22.035000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4d7e2-e98f-44d1-8d87-40b2c49237c2 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:22.046625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4d7e2-e98f-44d1-8d87-40b2c49237c2 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:22.052332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0e35e-63b2-4c94-980f-b7648ea93445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:22.054034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0e35e-63b2-4c94-980f-b7648ea93445 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:22.054434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0e35e-63b2-4c94-980f-b7648ea93445 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:22.054805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0e35e-63b2-4c94-980f-b7648ea93445 admin admin] PATH_INFO: `/v3/projects/ca6d3453f8ab40e9bab8c4c544c30533/users/79649c90970e40aaa1ff17875381cfff/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:22.074886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0e35e-63b2-4c94-980f-b7648ea93445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6d3453f8ab40e9bab8c4c544c30533, user_id=79649c90970e40aaa1ff17875381cfff, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:22.077519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0e35e-63b2-4c94-980f-b7648ea93445 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:22.193484 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1218/2384] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:07:22 2022] PUT /identity/v3/projects/ca6d3453f8ab40e9bab8c4c544c30533/users/79649c90970e40aaa1ff17875381cfff/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:07:22.198000 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c9e4c-4781-470e-9d4a-b5abe8918659 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:22.211744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c9e4c-4781-470e-9d4a-b5abe8918659 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:22.255412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:07:22.325230 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6fb2a-8ed7-4384-8d27-b93bbbc533a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:22.326127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6fb2a-8ed7-4384-8d27-b93bbbc533a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:22.326600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6fb2a-8ed7-4384-8d27-b93bbbc533a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:22.327080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6fb2a-8ed7-4384-8d27-b93bbbc533a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:22.328342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6fb2a-8ed7-4384-8d27-b93bbbc533a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:22.331235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6fb2a-8ed7-4384-8d27-b93bbbc533a5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:22.353946 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1167/2385] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:07:22.359147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0e35e-63b2-4c94-980f-b7648ea93445 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:22.371455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0e35e-63b2-4c94-980f-b7648ea93445 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:22.377392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7ec83-3c70-491f-8a63-65deafb87f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:22.378318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7ec83-3c70-491f-8a63-65deafb87f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:22.378692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7ec83-3c70-491f-8a63-65deafb87f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:22.382891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7ec83-3c70-491f-8a63-65deafb87f62 admin admin] PATH_INFO: `/v3/projects/ca6d3453f8ab40e9bab8c4c544c30533/users/79649c90970e40aaa1ff17875381cfff/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:22.382891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7ec83-3c70-491f-8a63-65deafb87f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6d3453f8ab40e9bab8c4c544c30533, user_id=79649c90970e40aaa1ff17875381cfff, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:22.385068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7ec83-3c70-491f-8a63-65deafb87f62 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:22.401560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1219/2386] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:07:22 2022] PUT /identity/v3/projects/ca6d3453f8ab40e9bab8c4c544c30533/users/79649c90970e40aaa1ff17875381cfff/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:07:31.328114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3930-0b4a-453f-93cf-1e553e4c2167 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:31.329760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3930-0b4a-453f-93cf-1e553e4c2167 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:31.330484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3930-0b4a-453f-93cf-1e553e4c2167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:31.378292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-55bc3930-0b4a-453f-93cf-1e553e4c2167 None None] MFA Rules not processed for user `79649c90970e40aaa1ff17875381cfff`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:07:31.379817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-55bc3930-0b4a-453f-93cf-1e553e4c2167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:31.514597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1168/2387] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:07:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:07:31.533561 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7ec83-3c70-491f-8a63-65deafb87f62 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:31.538509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-45b7ec83-3c70-491f-8a63-65deafb87f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:31.680791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7ec83-3c70-491f-8a63-65deafb87f62 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:31.690487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a485b-8508-4924-9943-c5e90b787657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:31.691304 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a485b-8508-4924-9943-c5e90b787657 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:31.691676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a485b-8508-4924-9943-c5e90b787657 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:31.692209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a485b-8508-4924-9943-c5e90b787657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:31.707061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:07:31.826386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a485b-8508-4924-9943-c5e90b787657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:31.828297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a485b-8508-4924-9943-c5e90b787657 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:31.840286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1220/2388] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:07:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:07:32.322960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc3930-0b4a-453f-93cf-1e553e4c2167 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:32.325297 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-55bc3930-0b4a-453f-93cf-1e553e4c2167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:32.410098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc3930-0b4a-453f-93cf-1e553e4c2167 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:32.416999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d36f3-1767-48b5-ab4f-9dd716c227f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:32.418311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d36f3-1767-48b5-ab4f-9dd716c227f2 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:32.418836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d36f3-1767-48b5-ab4f-9dd716c227f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:32.420228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d36f3-1767-48b5-ab4f-9dd716c227f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:32.423981 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-129d36f3-1767-48b5-ab4f-9dd716c227f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:32.454751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d36f3-1767-48b5-ab4f-9dd716c227f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:32.457228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d36f3-1767-48b5-ab4f-9dd716c227f2 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:32.472686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1169/2389] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:07:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:07:36.318151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec615ba-3373-42ff-a509-038b2729160b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:36.319734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec615ba-3373-42ff-a509-038b2729160b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:36.319734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec615ba-3373-42ff-a509-038b2729160b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:36.344439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-7ec615ba-3373-42ff-a509-038b2729160b None None] MFA Rules not processed for user `c7340e91e66f41559bb41e620ad3229f`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:07:36.345699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7ec615ba-3373-42ff-a509-038b2729160b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:36.353903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1221/2390] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:07:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:07:38.098523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d36f3-1767-48b5-ab4f-9dd716c227f2 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:38.165366 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:07:38.169069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d36f3-1767-48b5-ab4f-9dd716c227f2 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:38.173503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf135a9-5fec-4d50-889d-17c0a6c0ab3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:38.177639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf135a9-5fec-4d50-889d-17c0a6c0ab3c service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:38.177639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf135a9-5fec-4d50-889d-17c0a6c0ab3c service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:38.177639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf135a9-5fec-4d50-889d-17c0a6c0ab3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:38.177639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-fcf135a9-5fec-4d50-889d-17c0a6c0ab3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:38.198079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf135a9-5fec-4d50-889d-17c0a6c0ab3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:38.199908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf135a9-5fec-4d50-889d-17c0a6c0ab3c service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:38.213477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1170/2391] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:07:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:07:46.336685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=27 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:07:47.278984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bfb35-f4c7-416e-ac54-7a8f6bb1f7b5 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:47.279478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bfb35-f4c7-416e-ac54-7a8f6bb1f7b5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:47.279957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bfb35-f4c7-416e-ac54-7a8f6bb1f7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:47.313639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-cc2bfb35-f4c7-416e-ac54-7a8f6bb1f7b5 None None] MFA Rules not processed for user `0e8d03fe51534f8e867d0f83c3db2d36`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:07:47.315256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cc2bfb35-f4c7-416e-ac54-7a8f6bb1f7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:47.401358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1222/2392] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:07:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:07:48.207756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:07:48.448902 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf135a9-5fec-4d50-889d-17c0a6c0ab3c service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:07:48.454597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-fcf135a9-5fec-4d50-889d-17c0a6c0ab3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:48.476540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf135a9-5fec-4d50-889d-17c0a6c0ab3c service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:07:48.486588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6be1e1-4d25-4408-a6db-07bb8a5625c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:07:48.487554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6be1e1-4d25-4408-a6db-07bb8a5625c4 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:07:48.488228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6be1e1-4d25-4408-a6db-07bb8a5625c4 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:07:48.494299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6be1e1-4d25-4408-a6db-07bb8a5625c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:07:48.498060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0e6be1e1-4d25-4408-a6db-07bb8a5625c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:07:48.517886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6be1e1-4d25-4408-a6db-07bb8a5625c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:07:48.520863 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6be1e1-4d25-4408-a6db-07bb8a5625c4 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:07:48.538992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1171/2393] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:07:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:07:57.384994 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=25 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:07:58.530410 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=6 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:08:01.854983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bfb35-f4c7-416e-ac54-7a8f6bb1f7b5 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:08:01.866229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bfb35-f4c7-416e-ac54-7a8f6bb1f7b5 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:08:01.870171 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a28899-9a62-40ba-85c6-4b577eba934e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:08:01.870824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a28899-9a62-40ba-85c6-4b577eba934e service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:08:01.871107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a28899-9a62-40ba-85c6-4b577eba934e service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:08:01.871383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a28899-9a62-40ba-85c6-4b577eba934e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:08:01.873265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-41a28899-9a62-40ba-85c6-4b577eba934e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:08:01.884549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a28899-9a62-40ba-85c6-4b577eba934e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:08:01.886042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a28899-9a62-40ba-85c6-4b577eba934e service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:08:01.896108 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1223/2394] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:08:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:08:11.891282 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=6 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:08:17.040600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6be1e1-4d25-4408-a6db-07bb8a5625c4 service nova] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:08:17.042111 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0e6be1e1-4d25-4408-a6db-07bb8a5625c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:08:17.055425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6be1e1-4d25-4408-a6db-07bb8a5625c4 service nova] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:08:17.059256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56105e19-7bdc-41b5-afeb-6f7ecb05de97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:08:17.059884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56105e19-7bdc-41b5-afeb-6f7ecb05de97 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:08:17.060222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56105e19-7bdc-41b5-afeb-6f7ecb05de97 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:08:17.060493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56105e19-7bdc-41b5-afeb-6f7ecb05de97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:08:17.062448 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-56105e19-7bdc-41b5-afeb-6f7ecb05de97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:08:17.134088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56105e19-7bdc-41b5-afeb-6f7ecb05de97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:08:17.135559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56105e19-7bdc-41b5-afeb-6f7ecb05de97 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:08:17.146685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1172/2395] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:08:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:08:22.482569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a28899-9a62-40ba-85c6-4b577eba934e service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:08:22.509234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a28899-9a62-40ba-85c6-4b577eba934e service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:08:22.520229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f221bba-cd5e-4bf4-8692-c301349d0e1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:08:22.521347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f221bba-cd5e-4bf4-8692-c301349d0e1d service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:08:22.521826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f221bba-cd5e-4bf4-8692-c301349d0e1d service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:08:22.522295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f221bba-cd5e-4bf4-8692-c301349d0e1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:08:22.525475 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5f221bba-cd5e-4bf4-8692-c301349d0e1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:08:22.545597 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f221bba-cd5e-4bf4-8692-c301349d0e1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:08:22.548462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f221bba-cd5e-4bf4-8692-c301349d0e1d service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:08:22.568146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1224/2396] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:08:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:08:27.141098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=28 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:08:27.723411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559475b1-3196-4e90-b24b-8cebd7cf431e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:08:27.723715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559475b1-3196-4e90-b24b-8cebd7cf431e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:08:27.723987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559475b1-3196-4e90-b24b-8cebd7cf431e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:08:27.746178 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-559475b1-3196-4e90-b24b-8cebd7cf431e None None] MFA Rules not processed for user `fd040e13fd6c4b0dbea6a65932f84cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:08:27.747614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-559475b1-3196-4e90-b24b-8cebd7cf431e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:08:27.755957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1173/2397] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:08:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:08:28.137124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f221bba-cd5e-4bf4-8692-c301349d0e1d service nova] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:08:28.139243 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5f221bba-cd5e-4bf4-8692-c301349d0e1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:08:28.155002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f221bba-cd5e-4bf4-8692-c301349d0e1d service nova] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:08:28.159101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10672523-3c1a-4890-bb16-6a6fea85562b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:08:28.159764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10672523-3c1a-4890-bb16-6a6fea85562b service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:08:28.160046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10672523-3c1a-4890-bb16-6a6fea85562b service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:08:28.160393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10672523-3c1a-4890-bb16-6a6fea85562b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:08:28.162236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-10672523-3c1a-4890-bb16-6a6fea85562b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:08:28.173164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10672523-3c1a-4890-bb16-6a6fea85562b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:08:28.174643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10672523-3c1a-4890-bb16-6a6fea85562b service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:08:28.184246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1225/2398] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:08:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:08:28.709920 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559475b1-3196-4e90-b24b-8cebd7cf431e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:08:28.711443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-559475b1-3196-4e90-b24b-8cebd7cf431e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:08:28.872613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559475b1-3196-4e90-b24b-8cebd7cf431e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:08:28.879125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0af98-83f8-4c83-9ae1-2e9304985259 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:08:28.879766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0af98-83f8-4c83-9ae1-2e9304985259 service placement] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:08:28.880053 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0af98-83f8-4c83-9ae1-2e9304985259 service placement] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:08:28.880388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0af98-83f8-4c83-9ae1-2e9304985259 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:08:28.882367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3ad0af98-83f8-4c83-9ae1-2e9304985259 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:08:28.896592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0af98-83f8-4c83-9ae1-2e9304985259 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:08:28.898274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0af98-83f8-4c83-9ae1-2e9304985259 service placement] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:08:28.910743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1174/2399] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:08:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:08:38.179810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=12 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:08:38.905090 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=35 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:08:41.693571 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10672523-3c1a-4890-bb16-6a6fea85562b service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:08:41.740424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10672523-3c1a-4890-bb16-6a6fea85562b service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:08:41.746870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ea063-0a14-4b1e-ab2e-fdf8c0c3aef0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:08:41.747659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ea063-0a14-4b1e-ab2e-fdf8c0c3aef0 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:08:41.748058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ea063-0a14-4b1e-ab2e-fdf8c0c3aef0 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:08:41.748510 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ea063-0a14-4b1e-ab2e-fdf8c0c3aef0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:08:41.750708 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-180ea063-0a14-4b1e-ab2e-fdf8c0c3aef0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:08:41.776716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ea063-0a14-4b1e-ab2e-fdf8c0c3aef0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:08:41.778437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ea063-0a14-4b1e-ab2e-fdf8c0c3aef0 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:08:41.792106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1226/2400] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:08:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:08:51.785997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=21 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:09:02.745590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708d06c-7970-4044-b864-1c85a4175b3d None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:02.746526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708d06c-7970-4044-b864-1c85a4175b3d None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:02.746526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708d06c-7970-4044-b864-1c85a4175b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:02.771488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-0708d06c-7970-4044-b864-1c85a4175b3d None None] MFA Rules not processed for user `f3f607e48e534dd09c3619982978e9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:09:02.773054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0708d06c-7970-4044-b864-1c85a4175b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:02.891825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1175/2401] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:09:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:09:07.141127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ea063-0a14-4b1e-ab2e-fdf8c0c3aef0 service nova] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:07.192400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ea063-0a14-4b1e-ab2e-fdf8c0c3aef0 service nova] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:07.200452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc908d3c-cfb0-454a-8e9a-bee9ca64a72b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:07.201275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc908d3c-cfb0-454a-8e9a-bee9ca64a72b service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:07.201820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc908d3c-cfb0-454a-8e9a-bee9ca64a72b service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:07.202091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc908d3c-cfb0-454a-8e9a-bee9ca64a72b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:07.205333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-dc908d3c-cfb0-454a-8e9a-bee9ca64a72b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:07.243814 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc908d3c-cfb0-454a-8e9a-bee9ca64a72b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:07.247380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc908d3c-cfb0-454a-8e9a-bee9ca64a72b service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:07.277781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1227/2402] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:09:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:09:12.884677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=25 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:09:14.149809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708d06c-7970-4044-b864-1c85a4175b3d None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:14.281594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708d06c-7970-4044-b864-1c85a4175b3d None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:14.290171 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:14.291302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:14.292093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:14.293007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] PATH_INFO: `/v3/users/c7340e91e66f41559bb41e620ad3229f` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:14.295290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7340e91e66f41559bb41e620ad3229f)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:14.299604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:14.580638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7340e91e66f41559bb41e620ad3229f'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:14.592936 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7340e91e66f41559bb41e620ad3229f'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:14.594092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7340e91e66f41559bb41e620ad3229f'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:14.714779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7340e91e66f41559bb41e620ad3229f'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:14.733378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7340e91e66f41559bb41e620ad3229f'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:14.742190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1176/2403] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:09:14 2022] DELETE /identity/v3/users/c7340e91e66f41559bb41e620ad3229f => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:09:15.160894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc908d3c-cfb0-454a-8e9a-bee9ca64a72b service nova] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:15.172879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc908d3c-cfb0-454a-8e9a-bee9ca64a72b service nova] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:15.251214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:15.252134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:15.252450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:15.252727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] PATH_INFO: `/v3/projects/037b4d6af496475da87a47a9fd8d438d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:15.253963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=037b4d6af496475da87a47a9fd8d438d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:15.255922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:15.411019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '037b4d6af496475da87a47a9fd8d438d', 'request_id': 'req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:15.417753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '037b4d6af496475da87a47a9fd8d438d', 'request_id': 'req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:15.548624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '037b4d6af496475da87a47a9fd8d438d', 'request_id': 'req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:15.579741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] The token cache is being invalidate because project 037b4d6af496475da87a47a9fd8d438d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:09:15.580067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:15.580988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:15.582687 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1228/2404] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:09:15 2022] DELETE /identity/v3/projects/037b4d6af496475da87a47a9fd8d438d => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:09:15.588947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:15.591431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:15.615704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb617317-77d6-4d85-88ee-814e108298d3 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:15.724909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:15.725973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:15.726466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:15.727008 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] PATH_INFO: `/v3/users/79649c90970e40aaa1ff17875381cfff` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:15.729180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79649c90970e40aaa1ff17875381cfff)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:15.733195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:15.739969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:09:16.022712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79649c90970e40aaa1ff17875381cfff'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:16.031187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79649c90970e40aaa1ff17875381cfff'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:16.031985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79649c90970e40aaa1ff17875381cfff'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:16.126097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79649c90970e40aaa1ff17875381cfff'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:16.145201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-b0f63f85-644d-470e-b58e-487e57b97874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79649c90970e40aaa1ff17875381cfff'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:16.152396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1177/2405] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:09:15 2022] DELETE /identity/v3/users/79649c90970e40aaa1ff17875381cfff => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:09:16.464189 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:16.466205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:16.490241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6a8cb-77de-44a6-b721-86ecdf3c2ebd admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:16.566379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:09:16.579644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:16.580348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:16.580630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:16.580904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] PATH_INFO: `/v3/projects/ca6d3453f8ab40e9bab8c4c544c30533` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:16.582644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6d3453f8ab40e9bab8c4c544c30533)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:16.585849 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:16.764774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6d3453f8ab40e9bab8c4c544c30533', 'request_id': 'req-03d932f3-8813-408e-9cf5-de09df9a3bde'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:16.771981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6d3453f8ab40e9bab8c4c544c30533', 'request_id': 'req-03d932f3-8813-408e-9cf5-de09df9a3bde'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:16.880699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6d3453f8ab40e9bab8c4c544c30533', 'request_id': 'req-03d932f3-8813-408e-9cf5-de09df9a3bde'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:16.895797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] The token cache is being invalidate because project ca6d3453f8ab40e9bab8c4c544c30533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:09:16.896086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:16.896796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:16.898368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1229/2406] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:09:16 2022] DELETE /identity/v3/projects/ca6d3453f8ab40e9bab8c4c544c30533 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:09:23.073953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c0de2-0ef1-4b3f-8487-84c3701e1bf5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:23.074612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c0de2-0ef1-4b3f-8487-84c3701e1bf5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:23.074997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c0de2-0ef1-4b3f-8487-84c3701e1bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:23.131983 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-3e1c0de2-0ef1-4b3f-8487-84c3701e1bf5 None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:09:23.134211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3e1c0de2-0ef1-4b3f-8487-84c3701e1bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:23.250692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1178/2407] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 4 12:09:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:09:23.467931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:23.470930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:23.604073 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d932f3-8813-408e-9cf5-de09df9a3bde admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:23.613216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd1260-c5ad-4fce-a7b6-783447fd7607 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:23.614297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd1260-c5ad-4fce-a7b6-783447fd7607 service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:23.614800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd1260-c5ad-4fce-a7b6-783447fd7607 service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:23.615296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd1260-c5ad-4fce-a7b6-783447fd7607 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:23.630949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd1260-c5ad-4fce-a7b6-783447fd7607 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:23.633575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd1260-c5ad-4fce-a7b6-783447fd7607 service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:23.646321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1230/2408] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:09:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:09:28.416782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad1d31-6966-4e2f-9811-d23d077ac759 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:28.417449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad1d31-6966-4e2f-9811-d23d077ac759 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:28.417449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad1d31-6966-4e2f-9811-d23d077ac759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:28.441750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-54ad1d31-6966-4e2f-9811-d23d077ac759 None None] MFA Rules not processed for user `0e8d03fe51534f8e867d0f83c3db2d36`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:09:28.442868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-54ad1d31-6966-4e2f-9811-d23d077ac759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:28.533781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1179/2409] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:09:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:09:33.641169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:09:35.511500 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd1260-c5ad-4fce-a7b6-783447fd7607 service swift] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:35.515389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-70cd1260-c5ad-4fce-a7b6-783447fd7607 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:35.703455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd1260-c5ad-4fce-a7b6-783447fd7607 service swift] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:35.703455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df00d9-4d80-42ee-85a1-97db695f2a06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:35.703455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df00d9-4d80-42ee-85a1-97db695f2a06 service placement] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:35.703455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df00d9-4d80-42ee-85a1-97db695f2a06 service placement] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:35.703934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df00d9-4d80-42ee-85a1-97db695f2a06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:35.703934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d9df00d9-4d80-42ee-85a1-97db695f2a06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:35.717119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df00d9-4d80-42ee-85a1-97db695f2a06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:35.720058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df00d9-4d80-42ee-85a1-97db695f2a06 service placement] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:35.741362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1231/2410] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:09:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:09:35.915157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad1d31-6966-4e2f-9811-d23d077ac759 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:35.917439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-54ad1d31-6966-4e2f-9811-d23d077ac759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:35.957997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:09:36.014616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad1d31-6966-4e2f-9811-d23d077ac759 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:36.020472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:36.021273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:36.021556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:36.021837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] PATH_INFO: `/v3/users/2b8dc16b63584a46b9f63b52aa5ce391` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:36.023166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b8dc16b63584a46b9f63b52aa5ce391)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:36.025367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:36.361143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b8dc16b63584a46b9f63b52aa5ce391'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:36.369768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b8dc16b63584a46b9f63b52aa5ce391'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:36.370525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b8dc16b63584a46b9f63b52aa5ce391'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:36.471016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b8dc16b63584a46b9f63b52aa5ce391'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:36.486046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b8dc16b63584a46b9f63b52aa5ce391'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:36.492890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1180/2411] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:09:35 2022] DELETE /identity/v3/users/2b8dc16b63584a46b9f63b52aa5ce391 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:09:36.860150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df00d9-4d80-42ee-85a1-97db695f2a06 service placement] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:36.878331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df00d9-4d80-42ee-85a1-97db695f2a06 service placement] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:36.973920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:36.974689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:36.975073 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:36.975347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] PATH_INFO: `/v3/projects/950b28fc6faf434482bbb1c2db3fa194` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:36.976840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=950b28fc6faf434482bbb1c2db3fa194)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:36.978977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:37.180816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950b28fc6faf434482bbb1c2db3fa194', 'request_id': 'req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:37.187504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950b28fc6faf434482bbb1c2db3fa194', 'request_id': 'req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:37.328356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950b28fc6faf434482bbb1c2db3fa194', 'request_id': 'req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:37.342411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] The token cache is being invalidate because project 950b28fc6faf434482bbb1c2db3fa194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:09:37.342634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:37.343332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:37.345921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1232/2412] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:09:36 2022] DELETE /identity/v3/projects/950b28fc6faf434482bbb1c2db3fa194 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:09:37.350278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:37.358652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:37.365844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055763dd-c1df-4408-95d5-acc10ceeb373 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:37.457658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:37.458807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:37.459314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:37.459822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] PATH_INFO: `/v3/users/53d15c73206044a2a1d7418587018653` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:37.462134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d15c73206044a2a1d7418587018653)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:37.465990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:37.733795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d15c73206044a2a1d7418587018653'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:37.741881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d15c73206044a2a1d7418587018653'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:37.742639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d15c73206044a2a1d7418587018653'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:37.828698 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d15c73206044a2a1d7418587018653'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:37.843912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-6eb5407d-f465-4a93-8785-ab68cbedd9ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d15c73206044a2a1d7418587018653'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:37.850617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1181/2413] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:09:37 2022] DELETE /identity/v3/users/53d15c73206044a2a1d7418587018653 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:09:37.950334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:09:38.174336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:38.175874 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:38.210073 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51491b-ff3d-4a8f-9d61-188fb5c0bd89 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:38.237947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:09:38.285837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:38.286406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:38.286688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:38.286969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] PATH_INFO: `/v3/projects/1430f7af3a724f22b7bc770393490635` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:38.288215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1430f7af3a724f22b7bc770393490635)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:38.290442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:38.439542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1430f7af3a724f22b7bc770393490635', 'request_id': 'req-b6503f1a-cc28-4e9b-91e1-4610515de7f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:38.445691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1430f7af3a724f22b7bc770393490635', 'request_id': 'req-b6503f1a-cc28-4e9b-91e1-4610515de7f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:38.585216 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1430f7af3a724f22b7bc770393490635', 'request_id': 'req-b6503f1a-cc28-4e9b-91e1-4610515de7f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:38.599308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] The token cache is being invalidate because project 1430f7af3a724f22b7bc770393490635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:09:38.599607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:38.600320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:09:38.602024 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1233/2414] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:09:38 2022] DELETE /identity/v3/projects/1430f7af3a724f22b7bc770393490635 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:09:48.659251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba6a5d-2b27-4097-bdd5-3eaa55993df1 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:48.660754 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba6a5d-2b27-4097-bdd5-3eaa55993df1 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:48.661256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba6a5d-2b27-4097-bdd5-3eaa55993df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:48.696556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e1ba6a5d-2b27-4097-bdd5-3eaa55993df1 None None] MFA Rules not processed for user `76771369a0df4c5189e42291ebdf9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:09:48.698664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e1ba6a5d-2b27-4097-bdd5-3eaa55993df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:48.785891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1182/2415] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:09:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:09:48.954670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:09:58.772876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:DELETE=4,keystone:SELECT=38,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:09:59.598482 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:09:59.599944 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:59.705616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6503f1a-cc28-4e9b-91e1-4610515de7f6 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:09:59.711064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f09883-6060-4219-bf43-a962aa76ed2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:09:59.716239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f09883-6060-4219-bf43-a962aa76ed2a service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:09:59.716239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f09883-6060-4219-bf43-a962aa76ed2a service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:09:59.716239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f09883-6060-4219-bf43-a962aa76ed2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:09:59.716239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f5f09883-6060-4219-bf43-a962aa76ed2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:09:59.815345 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f09883-6060-4219-bf43-a962aa76ed2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:09:59.818182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f09883-6060-4219-bf43-a962aa76ed2a service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:09:59.833366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1234/2416] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:09:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:10:09.828133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=57 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:31.419937 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba6a5d-2b27-4097-bdd5-3eaa55993df1 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:31.421605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e1ba6a5d-2b27-4097-bdd5-3eaa55993df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:31.526171 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba6a5d-2b27-4097-bdd5-3eaa55993df1 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:31.531660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d80a4-a32c-4e17-ad46-311a698f1a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:31.532584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d80a4-a32c-4e17-ad46-311a698f1a4d service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:31.532965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d80a4-a32c-4e17-ad46-311a698f1a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:31.533443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d80a4-a32c-4e17-ad46-311a698f1a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:31.540404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8b1d80a4-a32c-4e17-ad46-311a698f1a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:31.624476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d80a4-a32c-4e17-ad46-311a698f1a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:31.625995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d80a4-a32c-4e17-ad46-311a698f1a4d service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:31.638601 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1183/2417] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:10:34.131722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f09883-6060-4219-bf43-a962aa76ed2a service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:34.134370 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f5f09883-6060-4219-bf43-a962aa76ed2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:34.149888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f09883-6060-4219-bf43-a962aa76ed2a service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:34.155592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab424e1-f670-45e4-9f3e-8f3f4812d4ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:34.156517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab424e1-f670-45e4-9f3e-8f3f4812d4ed service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:34.157034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab424e1-f670-45e4-9f3e-8f3f4812d4ed service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:34.157502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab424e1-f670-45e4-9f3e-8f3f4812d4ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:34.160770 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6ab424e1-f670-45e4-9f3e-8f3f4812d4ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:34.186577 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab424e1-f670-45e4-9f3e-8f3f4812d4ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:34.189965 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab424e1-f670-45e4-9f3e-8f3f4812d4ed service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:34.215723 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1235/2418] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:10:34.665158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d80a4-a32c-4e17-ad46-311a698f1a4d service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:34.666927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8b1d80a4-a32c-4e17-ad46-311a698f1a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:34.784987 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d80a4-a32c-4e17-ad46-311a698f1a4d service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:34.795639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cef4d-c97e-4839-be0e-14506c3a3f1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:34.797556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cef4d-c97e-4839-be0e-14506c3a3f1a service placement] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:34.797556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cef4d-c97e-4839-be0e-14506c3a3f1a service placement] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:34.797556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cef4d-c97e-4839-be0e-14506c3a3f1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:34.800401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f49cef4d-c97e-4839-be0e-14506c3a3f1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:34.816950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cef4d-c97e-4839-be0e-14506c3a3f1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:34.818910 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cef4d-c97e-4839-be0e-14506c3a3f1a service placement] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:34.827198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:34.834784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1184/2419] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:10:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:10:37.397115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab424e1-f670-45e4-9f3e-8f3f4812d4ed service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:37.400684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6ab424e1-f670-45e4-9f3e-8f3f4812d4ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:37.475798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab424e1-f670-45e4-9f3e-8f3f4812d4ed service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:37.488148 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:37.489722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:37.490509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:37.491010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] PATH_INFO: `/v3/users/fd040e13fd6c4b0dbea6a65932f84cb2` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:37.492983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd040e13fd6c4b0dbea6a65932f84cb2)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:37.494999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:37.779139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd040e13fd6c4b0dbea6a65932f84cb2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:37.779139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd040e13fd6c4b0dbea6a65932f84cb2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:37.793677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd040e13fd6c4b0dbea6a65932f84cb2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:37.798879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd040e13fd6c4b0dbea6a65932f84cb2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:37.927019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-3d7795dd-e73f-49d2-af12-0224c0615210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd040e13fd6c4b0dbea6a65932f84cb2'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:37.947001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1236/2420] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:10:37 2022] DELETE /identity/v3/users/fd040e13fd6c4b0dbea6a65932f84cb2 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:10:38.326485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cef4d-c97e-4839-be0e-14506c3a3f1a service placement] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:38.342818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cef4d-c97e-4839-be0e-14506c3a3f1a service placement] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:38.411661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:38.414499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:38.414499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:38.414499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] PATH_INFO: `/v3/projects/4eb043e789a6467db5465a0107f41076` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:38.414499 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eb043e789a6467db5465a0107f41076)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:38.416029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:38.598585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eb043e789a6467db5465a0107f41076', 'request_id': 'req-9c10c339-5ac0-42f6-960d-c8a9737fe47d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:38.605877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eb043e789a6467db5465a0107f41076', 'request_id': 'req-9c10c339-5ac0-42f6-960d-c8a9737fe47d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:38.615807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eb043e789a6467db5465a0107f41076', 'request_id': 'req-9c10c339-5ac0-42f6-960d-c8a9737fe47d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:38.751846 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] The token cache is being invalidate because project 4eb043e789a6467db5465a0107f41076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:10:38.752763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:38.753034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-9c10c339-5ac0-42f6-960d-c8a9737fe47d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:38.754782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1185/2421] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:10:38 2022] DELETE /identity/v3/projects/4eb043e789a6467db5465a0107f41076 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:10:38.760083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693f15b-853b-4988-a4b7-9d787b3bd218 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:38.760652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693f15b-853b-4988-a4b7-9d787b3bd218 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:38.760652 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693f15b-853b-4988-a4b7-9d787b3bd218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:38.847368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b693f15b-853b-4988-a4b7-9d787b3bd218 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:38.847368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b693f15b-853b-4988-a4b7-9d787b3bd218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:38.915155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1237/2422] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:10:38.925001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303db082-268f-4985-9df1-a53e9a1ef976 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:38.925373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303db082-268f-4985-9df1-a53e9a1ef976 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:38.925730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303db082-268f-4985-9df1-a53e9a1ef976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:38.965782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-303db082-268f-4985-9df1-a53e9a1ef976 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:38.966264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-303db082-268f-4985-9df1-a53e9a1ef976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:38.980779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1186/2423] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:10:38.987650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693f15b-853b-4988-a4b7-9d787b3bd218 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:39.011100 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693f15b-853b-4988-a4b7-9d787b3bd218 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:39.020867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d171c-5d52-4184-8097-8a4405547a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:39.021928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d171c-5d52-4184-8097-8a4405547a7c admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:39.022452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d171c-5d52-4184-8097-8a4405547a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:39.022947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d171c-5d52-4184-8097-8a4405547a7c admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:39.024000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d171c-5d52-4184-8097-8a4405547a7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:10:39.024412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d171c-5d52-4184-8097-8a4405547a7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:39.028912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d171c-5d52-4184-8097-8a4405547a7c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:39.043311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1238/2424] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:10:39.048050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303db082-268f-4985-9df1-a53e9a1ef976 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:39.057739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303db082-268f-4985-9df1-a53e9a1ef976 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:39.063882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4502d9b-f66e-4d28-9d16-df4c178bcbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:39.064720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4502d9b-f66e-4d28-9d16-df4c178bcbb7 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:39.065003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4502d9b-f66e-4d28-9d16-df4c178bcbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:39.065278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4502d9b-f66e-4d28-9d16-df4c178bcbb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:39.065932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4502d9b-f66e-4d28-9d16-df4c178bcbb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:39.067895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4502d9b-f66e-4d28-9d16-df4c178bcbb7 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:39.217264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1187/2425] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:10:39 2022] POST /identity/v3/projects => generated 367 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:10:39.220127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d171c-5d52-4184-8097-8a4405547a7c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:39.231966 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d171c-5d52-4184-8097-8a4405547a7c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:39.244852 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:39.312838 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d27a4-fb82-4107-b548-c7b959784377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:39.313460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d27a4-fb82-4107-b548-c7b959784377 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:39.313743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d27a4-fb82-4107-b548-c7b959784377 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:39.314047 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d27a4-fb82-4107-b548-c7b959784377 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:39.314681 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d27a4-fb82-4107-b548-c7b959784377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:39.316699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d27a4-fb82-4107-b548-c7b959784377 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:39.446802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1239/2426] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:10:39 2022] POST /identity/v3/users => generated 284 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:10:39.448917 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4502d9b-f66e-4d28-9d16-df4c178bcbb7 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:39.463195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4502d9b-f66e-4d28-9d16-df4c178bcbb7 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:39.469172 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700a1e5-537a-4002-b8bf-425f64c9be47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:39.469790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700a1e5-537a-4002-b8bf-425f64c9be47 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:39.470260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700a1e5-537a-4002-b8bf-425f64c9be47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:39.471291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700a1e5-537a-4002-b8bf-425f64c9be47 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:39.472429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700a1e5-537a-4002-b8bf-425f64c9be47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:39.474411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700a1e5-537a-4002-b8bf-425f64c9be47 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:39.499364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1188/2427] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:10:39.505729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d27a4-fb82-4107-b548-c7b959784377 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:39.531999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d27a4-fb82-4107-b548-c7b959784377 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:39.538481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e4b65-c915-4d58-a03a-19b8da7e6d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:39.539097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e4b65-c915-4d58-a03a-19b8da7e6d34 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:39.539381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e4b65-c915-4d58-a03a-19b8da7e6d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:39.539664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e4b65-c915-4d58-a03a-19b8da7e6d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:39.540381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e4b65-c915-4d58-a03a-19b8da7e6d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:39.542079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e4b65-c915-4d58-a03a-19b8da7e6d34 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:39.554609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1240/2428] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:10:39.561208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700a1e5-537a-4002-b8bf-425f64c9be47 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:39.572562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700a1e5-537a-4002-b8bf-425f64c9be47 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:39.580896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7349931-fa81-43d8-a00d-e4d330618d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:39.581821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7349931-fa81-43d8-a00d-e4d330618d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:39.582203 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7349931-fa81-43d8-a00d-e4d330618d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:39.582577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7349931-fa81-43d8-a00d-e4d330618d7a admin admin] PATH_INFO: `/v3/projects/f67b2888a1fc45b9bed0b15c87d537f4/users/55de5ec151644d29a9b87aceb2d34c4e/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:39.611526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7349931-fa81-43d8-a00d-e4d330618d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f67b2888a1fc45b9bed0b15c87d537f4, user_id=55de5ec151644d29a9b87aceb2d34c4e, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:39.617388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7349931-fa81-43d8-a00d-e4d330618d7a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:39.736243 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1189/2429] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:10:39 2022] PUT /identity/v3/projects/f67b2888a1fc45b9bed0b15c87d537f4/users/55de5ec151644d29a9b87aceb2d34c4e/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:10:39.754215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10213f-4660-41b5-81d5-a615a600e963 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:39.754492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10213f-4660-41b5-81d5-a615a600e963 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:39.754780 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10213f-4660-41b5-81d5-a615a600e963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:39.795704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-8b10213f-4660-41b5-81d5-a615a600e963 None None] MFA Rules not processed for user `55de5ec151644d29a9b87aceb2d34c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:39.798863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8b10213f-4660-41b5-81d5-a615a600e963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:39.966737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1241/2430] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:10:39.980939 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7349931-fa81-43d8-a00d-e4d330618d7a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:39.983038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b7349931-fa81-43d8-a00d-e4d330618d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:40.035332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:40.097557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7349931-fa81-43d8-a00d-e4d330618d7a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:40.101759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b93171-ff89-48c7-b9e3-58c153b7b88d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:40.102387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b93171-ff89-48c7-b9e3-58c153b7b88d service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:40.102663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b93171-ff89-48c7-b9e3-58c153b7b88d service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:40.102937 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b93171-ff89-48c7-b9e3-58c153b7b88d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:40.117382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b93171-ff89-48c7-b9e3-58c153b7b88d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:40.117382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b93171-ff89-48c7-b9e3-58c153b7b88d service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:40.131089 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1190/2431] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:10:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:10:40.277992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10213f-4660-41b5-81d5-a615a600e963 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:40.279527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8b10213f-4660-41b5-81d5-a615a600e963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:40.374841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10213f-4660-41b5-81d5-a615a600e963 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:40.380084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:40.380767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:40.386603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:40.386603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] PATH_INFO: `/v3/users/f3f607e48e534dd09c3619982978e9e5` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:40.386603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f607e48e534dd09c3619982978e9e5)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:40.386603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:40.527239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:40.630882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f607e48e534dd09c3619982978e9e5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:40.631559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f607e48e534dd09c3619982978e9e5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:40.639069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f607e48e534dd09c3619982978e9e5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:40.639760 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f607e48e534dd09c3619982978e9e5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:40.755269 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-304b76b9-0fc9-47cb-94ac-33762e797758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f607e48e534dd09c3619982978e9e5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:40.778144 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1242/2432] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:10:40 2022] DELETE /identity/v3/users/f3f607e48e534dd09c3619982978e9e5 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:10:41.056746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b93171-ff89-48c7-b9e3-58c153b7b88d service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:41.069887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b93171-ff89-48c7-b9e3-58c153b7b88d service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:41.131547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:41.132237 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:41.132495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:41.133449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] PATH_INFO: `/v3/projects/f4f180049cc049829e12eace469bef05` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:41.133957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f180049cc049829e12eace469bef05)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:41.143488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:41.287783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f180049cc049829e12eace469bef05', 'request_id': 'req-ac5f5370-26e3-4d7c-925e-6aefb53f123d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:41.294954 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f180049cc049829e12eace469bef05', 'request_id': 'req-ac5f5370-26e3-4d7c-925e-6aefb53f123d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:41.302695 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f180049cc049829e12eace469bef05', 'request_id': 'req-ac5f5370-26e3-4d7c-925e-6aefb53f123d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:41.438592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] The token cache is being invalidate because project f4f180049cc049829e12eace469bef05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:10:41.438889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:41.439784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ac5f5370-26e3-4d7c-925e-6aefb53f123d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:41.441435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1191/2433] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:10:41 2022] DELETE /identity/v3/projects/f4f180049cc049829e12eace469bef05 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:10:41.446718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786f04a-a45a-4814-aaa2-cf56297f14e3 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:41.447290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786f04a-a45a-4814-aaa2-cf56297f14e3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:41.447290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786f04a-a45a-4814-aaa2-cf56297f14e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:41.469679 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1786f04a-a45a-4814-aaa2-cf56297f14e3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:41.470914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1786f04a-a45a-4814-aaa2-cf56297f14e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:41.538901 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1243/2434] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:10:41.547762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cf11b-d229-4664-823e-687a695e2583 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:41.548044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cf11b-d229-4664-823e-687a695e2583 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:41.548353 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cf11b-d229-4664-823e-687a695e2583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:41.569759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-803cf11b-d229-4664-823e-687a695e2583 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:41.570993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-803cf11b-d229-4664-823e-687a695e2583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:41.580339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1192/2435] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:10:41.584734 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786f04a-a45a-4814-aaa2-cf56297f14e3 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:41.598068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786f04a-a45a-4814-aaa2-cf56297f14e3 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:41.605374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0afd075-a56d-4d06-97b3-436bd5383095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:41.606036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afd075-a56d-4d06-97b3-436bd5383095 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:41.606321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afd075-a56d-4d06-97b3-436bd5383095 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:41.606626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afd075-a56d-4d06-97b3-436bd5383095 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:41.607437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0afd075-a56d-4d06-97b3-436bd5383095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:10:41.607776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0afd075-a56d-4d06-97b3-436bd5383095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:41.609751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0afd075-a56d-4d06-97b3-436bd5383095 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:41.623959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1244/2436] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:10:41.629130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cf11b-d229-4664-823e-687a695e2583 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:41.639464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cf11b-d229-4664-823e-687a695e2583 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:41.644568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a04cc9-03c3-4a2c-a3f4-3fe34cc79622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:41.645294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04cc9-03c3-4a2c-a3f4-3fe34cc79622 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:41.645586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04cc9-03c3-4a2c-a3f4-3fe34cc79622 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:41.645872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04cc9-03c3-4a2c-a3f4-3fe34cc79622 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:41.646532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a04cc9-03c3-4a2c-a3f4-3fe34cc79622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:41.648719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a04cc9-03c3-4a2c-a3f4-3fe34cc79622 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:41.789216 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1193/2437] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:10:41 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:10:41.793763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0afd075-a56d-4d06-97b3-436bd5383095 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:41.807038 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0afd075-a56d-4d06-97b3-436bd5383095 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:41.897106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bb6d4-6451-4df1-9941-a73c9043d630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:41.897726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bb6d4-6451-4df1-9941-a73c9043d630 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:41.898009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bb6d4-6451-4df1-9941-a73c9043d630 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:41.898293 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bb6d4-6451-4df1-9941-a73c9043d630 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:41.898928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bb6d4-6451-4df1-9941-a73c9043d630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:41.901066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bb6d4-6451-4df1-9941-a73c9043d630 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:42.054249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1245/2438] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:10:41 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:10:42.061773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a04cc9-03c3-4a2c-a3f4-3fe34cc79622 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:42.073691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a04cc9-03c3-4a2c-a3f4-3fe34cc79622 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:42.079001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660f26c-b263-46e0-8a1b-db9befcb738e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:42.079619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660f26c-b263-46e0-8a1b-db9befcb738e admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:42.079894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660f26c-b263-46e0-8a1b-db9befcb738e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:42.080162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660f26c-b263-46e0-8a1b-db9befcb738e admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:42.080836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660f26c-b263-46e0-8a1b-db9befcb738e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:42.082527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660f26c-b263-46e0-8a1b-db9befcb738e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:42.094553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1194/2439] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:10:42.099107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bb6d4-6451-4df1-9941-a73c9043d630 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:42.111088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bb6d4-6451-4df1-9941-a73c9043d630 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:42.117397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f76ca-3a7b-4214-8d70-0f1568540c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:42.117950 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f76ca-3a7b-4214-8d70-0f1568540c29 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:42.118294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f76ca-3a7b-4214-8d70-0f1568540c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:42.118496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f76ca-3a7b-4214-8d70-0f1568540c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:42.119268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f76ca-3a7b-4214-8d70-0f1568540c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:42.120967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f76ca-3a7b-4214-8d70-0f1568540c29 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:42.134740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1246/2440] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:10:42.139513 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660f26c-b263-46e0-8a1b-db9befcb738e admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:42.150290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660f26c-b263-46e0-8a1b-db9befcb738e admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:42.155654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad1997-db69-4139-b937-7eb8d8aa06cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:42.156451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad1997-db69-4139-b937-7eb8d8aa06cd admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:42.156792 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad1997-db69-4139-b937-7eb8d8aa06cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:42.157132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad1997-db69-4139-b937-7eb8d8aa06cd admin admin] PATH_INFO: `/v3/projects/e41d4317234c458daad66f591d27c6fa/users/9ab97a61223f4719add7b3c0f845a319/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:42.167697 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:42.177083 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad1997-db69-4139-b937-7eb8d8aa06cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41d4317234c458daad66f591d27c6fa, user_id=9ab97a61223f4719add7b3c0f845a319, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:42.179854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad1997-db69-4139-b937-7eb8d8aa06cd admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:42.369577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1195/2441] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:10:42 2022] PUT /identity/v3/projects/e41d4317234c458daad66f591d27c6fa/users/9ab97a61223f4719add7b3c0f845a319/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:10:42.378020 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e714837-3742-494f-891d-e9c2894dda80 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:42.378476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e714837-3742-494f-891d-e9c2894dda80 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:42.378931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e714837-3742-494f-891d-e9c2894dda80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:42.408996 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-0e714837-3742-494f-891d-e9c2894dda80 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:42.410518 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-0e714837-3742-494f-891d-e9c2894dda80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:42.415625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:42.480932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1247/2442] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:10:42.492123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad1997-db69-4139-b937-7eb8d8aa06cd admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:42.494056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-54ad1997-db69-4139-b937-7eb8d8aa06cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:42.572644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad1997-db69-4139-b937-7eb8d8aa06cd admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:42.577026 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8d9a6-7b4c-4ebb-81c5-ca9f817ba1c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:42.577652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d9a6-7b4c-4ebb-81c5-ca9f817ba1c9 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:42.577914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d9a6-7b4c-4ebb-81c5-ca9f817ba1c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:42.578188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d9a6-7b4c-4ebb-81c5-ca9f817ba1c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:42.588542 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8d9a6-7b4c-4ebb-81c5-ca9f817ba1c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:42.589991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8d9a6-7b4c-4ebb-81c5-ca9f817ba1c9 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:42.600799 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1196/2443] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:10:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:10:51.100959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d1446-79c3-44cd-92e1-defb3febd581 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:51.103915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d1446-79c3-44cd-92e1-defb3febd581 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:51.103915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d1446-79c3-44cd-92e1-defb3febd581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:51.154190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-eb6d1446-79c3-44cd-92e1-defb3febd581 None None] MFA Rules not processed for user `9ab97a61223f4719add7b3c0f845a319`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:51.155912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-eb6d1446-79c3-44cd-92e1-defb3febd581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:51.282461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1248/2444] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:10:51.301941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8d9a6-7b4c-4ebb-81c5-ca9f817ba1c9 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:51.303969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e9c8d9a6-7b4c-4ebb-81c5-ca9f817ba1c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:51.452084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8d9a6-7b4c-4ebb-81c5-ca9f817ba1c9 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:51.459860 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eba5b-689b-4ac8-815a-26fc96b50a5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:51.460790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eba5b-689b-4ac8-815a-26fc96b50a5a service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:51.461562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eba5b-689b-4ac8-815a-26fc96b50a5a service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:51.461797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eba5b-689b-4ac8-815a-26fc96b50a5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:51.472740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6eba5b-689b-4ac8-815a-26fc96b50a5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:51.474552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6eba5b-689b-4ac8-815a-26fc96b50a5a service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:51.488822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1197/2445] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:10:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:10:53.435512 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d1446-79c3-44cd-92e1-defb3febd581 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:53.438499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-eb6d1446-79c3-44cd-92e1-defb3febd581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:53.457523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d1446-79c3-44cd-92e1-defb3febd581 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:53.464675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:53.481265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:53.481265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:53.481265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] PATH_INFO: `/v3/users/55de5ec151644d29a9b87aceb2d34c4e` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:53.481265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] RBAC: Authorizing `identity:delete_user(user_id=55de5ec151644d29a9b87aceb2d34c4e)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:53.481265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:53.925897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55de5ec151644d29a9b87aceb2d34c4e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:53.926746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55de5ec151644d29a9b87aceb2d34c4e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:53.937147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55de5ec151644d29a9b87aceb2d34c4e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:53.937959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55de5ec151644d29a9b87aceb2d34c4e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:54.116155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55de5ec151644d29a9b87aceb2d34c4e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:54.152856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1249/2446] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:10:53 2022] DELETE /identity/v3/users/55de5ec151644d29a9b87aceb2d34c4e => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:10:54.158933 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86192a-a40d-4a07-9894-26c1b0ebb40b None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:54.160720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86192a-a40d-4a07-9894-26c1b0ebb40b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:54.161780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86192a-a40d-4a07-9894-26c1b0ebb40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:54.212472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-0d86192a-a40d-4a07-9894-26c1b0ebb40b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:54.215224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0d86192a-a40d-4a07-9894-26c1b0ebb40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:54.301255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:54.340692 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1198/2447] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:10:54.354910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:54.361053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:54.397478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:54.488789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7afe17-1095-41a9-adaf-56fcbefc025c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:54.494371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59079ab-1c07-40ef-b342-eda2c84af404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:54.495015 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59079ab-1c07-40ef-b342-eda2c84af404 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:54.495308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59079ab-1c07-40ef-b342-eda2c84af404 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:54.495588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59079ab-1c07-40ef-b342-eda2c84af404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:54.516219 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59079ab-1c07-40ef-b342-eda2c84af404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:54.517908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59079ab-1c07-40ef-b342-eda2c84af404 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:54.544259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1250/2448] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:10:55.546480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86192a-a40d-4a07-9894-26c1b0ebb40b None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:55.549752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0d86192a-a40d-4a07-9894-26c1b0ebb40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:55.583582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86192a-a40d-4a07-9894-26c1b0ebb40b None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:55.606298 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:55.609267 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:55.609695 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:55.611582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] PATH_INFO: `/v3/projects/f67b2888a1fc45b9bed0b15c87d537f4` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:55.617800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f67b2888a1fc45b9bed0b15c87d537f4)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:55.624467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:55.892877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f67b2888a1fc45b9bed0b15c87d537f4', 'request_id': 'req-800c4315-a8a2-4b70-9d1c-5187f286dcaf'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:55.903612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f67b2888a1fc45b9bed0b15c87d537f4', 'request_id': 'req-800c4315-a8a2-4b70-9d1c-5187f286dcaf'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:55.920892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f67b2888a1fc45b9bed0b15c87d537f4', 'request_id': 'req-800c4315-a8a2-4b70-9d1c-5187f286dcaf'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:56.004449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88af7-99dc-4166-8900-0bbf68fa4006 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:56.004915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88af7-99dc-4166-8900-0bbf68fa4006 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:56.005308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88af7-99dc-4166-8900-0bbf68fa4006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:56.040719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] The token cache is being invalidate because project f67b2888a1fc45b9bed0b15c87d537f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:10:56.040719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:56.040719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-800c4315-a8a2-4b70-9d1c-5187f286dcaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:10:56.040979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1199/2449] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:10:55 2022] DELETE /identity/v3/projects/f67b2888a1fc45b9bed0b15c87d537f4 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:10:56.047223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece0924-fd52-4c69-a706-0dc8956531b4 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:56.047561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece0924-fd52-4c69-a706-0dc8956531b4 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:56.047797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece0924-fd52-4c69-a706-0dc8956531b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:56.056066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-cde88af7-99dc-4166-8900-0bbf68fa4006 None None] MFA Rules not processed for user `9ab97a61223f4719add7b3c0f845a319`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:56.061361 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cde88af7-99dc-4166-8900-0bbf68fa4006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:56.091257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-4ece0924-fd52-4c69-a706-0dc8956531b4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:56.092587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4ece0924-fd52-4c69-a706-0dc8956531b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:56.186858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1251/2450] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:10:56.224673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1200/2451] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:10:56.234133 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0e868-35e7-4fcc-a321-042947e4ac3b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:56.234442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0e868-35e7-4fcc-a321-042947e4ac3b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:56.234712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0e868-35e7-4fcc-a321-042947e4ac3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:56.281135 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f6f0e868-35e7-4fcc-a321-042947e4ac3b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:56.282408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f6f0e868-35e7-4fcc-a321-042947e4ac3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:56.297897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1252/2452] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:10:56.298338 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece0924-fd52-4c69-a706-0dc8956531b4 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:56.312281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece0924-fd52-4c69-a706-0dc8956531b4 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:56.324560 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f039d2-0c82-49af-ba57-7713bda42f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:56.325209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f039d2-0c82-49af-ba57-7713bda42f5d admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:56.326029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f039d2-0c82-49af-ba57-7713bda42f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:56.326029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f039d2-0c82-49af-ba57-7713bda42f5d admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:56.326473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f039d2-0c82-49af-ba57-7713bda42f5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:10:56.326823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f039d2-0c82-49af-ba57-7713bda42f5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:56.328738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f039d2-0c82-49af-ba57-7713bda42f5d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:56.348833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1201/2453] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:10:56.351008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0e868-35e7-4fcc-a321-042947e4ac3b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:56.366624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0e868-35e7-4fcc-a321-042947e4ac3b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:56.377550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79321f46-072e-475c-aff4-0f0f67b42302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:56.381282 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79321f46-072e-475c-aff4-0f0f67b42302 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:56.383895 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79321f46-072e-475c-aff4-0f0f67b42302 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:56.385029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79321f46-072e-475c-aff4-0f0f67b42302 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:56.389334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79321f46-072e-475c-aff4-0f0f67b42302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:56.390633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79321f46-072e-475c-aff4-0f0f67b42302 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:56.540327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1253/2454] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:10:56 2022] POST /identity/v3/projects => generated 381 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:10:56.547497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f039d2-0c82-49af-ba57-7713bda42f5d admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:56.561269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f039d2-0c82-49af-ba57-7713bda42f5d admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:56.650155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183c5a6-724b-4388-b183-f2c05ffa8734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:56.650853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183c5a6-724b-4388-b183-f2c05ffa8734 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:56.651133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183c5a6-724b-4388-b183-f2c05ffa8734 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:56.651537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183c5a6-724b-4388-b183-f2c05ffa8734 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:56.652348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183c5a6-724b-4388-b183-f2c05ffa8734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:56.655253 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183c5a6-724b-4388-b183-f2c05ffa8734 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:56.664333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:56.902832 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1202/2455] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:10:56 2022] POST /identity/v3/users => generated 291 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:10:56.907065 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79321f46-072e-475c-aff4-0f0f67b42302 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:56.922463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79321f46-072e-475c-aff4-0f0f67b42302 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:56.931094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0b4ec-f3aa-4550-bc93-03e850859cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:56.932112 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0b4ec-f3aa-4550-bc93-03e850859cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:56.932440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0b4ec-f3aa-4550-bc93-03e850859cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:56.932803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0b4ec-f3aa-4550-bc93-03e850859cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:56.935909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0b4ec-f3aa-4550-bc93-03e850859cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:56.938776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0b4ec-f3aa-4550-bc93-03e850859cb8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:56.958956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1254/2456] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:10:56.965719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183c5a6-724b-4388-b183-f2c05ffa8734 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:56.976048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183c5a6-724b-4388-b183-f2c05ffa8734 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:56.981640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5235e61-3006-4c0d-896b-7327af6a1c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:56.982400 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5235e61-3006-4c0d-896b-7327af6a1c74 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:56.982722 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5235e61-3006-4c0d-896b-7327af6a1c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:56.983032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5235e61-3006-4c0d-896b-7327af6a1c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:56.983817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5235e61-3006-4c0d-896b-7327af6a1c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:56.985567 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5235e61-3006-4c0d-896b-7327af6a1c74 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:56.997455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1203/2457] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:10:57.003757 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0b4ec-f3aa-4550-bc93-03e850859cb8 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:57.013951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0b4ec-f3aa-4550-bc93-03e850859cb8 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:57.018607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ac792-0460-49a5-8ed8-05a34ced601a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:57.019268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ac792-0460-49a5-8ed8-05a34ced601a admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:57.019545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ac792-0460-49a5-8ed8-05a34ced601a admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:57.019818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ac792-0460-49a5-8ed8-05a34ced601a admin admin] PATH_INFO: `/v3/projects/90111ff0ed744a86aed1ed3c59f53dfe/users/a11dd1e91e024801a91a1f456fdf68db/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:57.038619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ac792-0460-49a5-8ed8-05a34ced601a admin admin] RBAC: Authorizing `identity:create_grant(project_id=90111ff0ed744a86aed1ed3c59f53dfe, user_id=a11dd1e91e024801a91a1f456fdf68db, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:57.041469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ac792-0460-49a5-8ed8-05a34ced601a admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:57.192931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1255/2458] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:10:57 2022] PUT /identity/v3/projects/90111ff0ed744a86aed1ed3c59f53dfe/users/a11dd1e91e024801a91a1f456fdf68db/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:10:57.205060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c250f-90cf-4c16-81a7-d992eaa31caa None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:57.205356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c250f-90cf-4c16-81a7-d992eaa31caa None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:57.205634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c250f-90cf-4c16-81a7-d992eaa31caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:57.238813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e85c250f-90cf-4c16-81a7-d992eaa31caa None None] MFA Rules not processed for user `a11dd1e91e024801a91a1f456fdf68db`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:57.240060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e85c250f-90cf-4c16-81a7-d992eaa31caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:57.318941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1204/2459] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:10:57.323859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ac792-0460-49a5-8ed8-05a34ced601a admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:57.411396 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:57.417412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ac792-0460-49a5-8ed8-05a34ced601a admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:57.422751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01206c8c-0149-432c-95ad-17aab3e8aec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:57.423383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01206c8c-0149-432c-95ad-17aab3e8aec3 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:57.423658 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01206c8c-0149-432c-95ad-17aab3e8aec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:57.423929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01206c8c-0149-432c-95ad-17aab3e8aec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:57.424737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01206c8c-0149-432c-95ad-17aab3e8aec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:57.426577 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01206c8c-0149-432c-95ad-17aab3e8aec3 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:57.563751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1256/2460] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:10:57 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:10:57.570285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c250f-90cf-4c16-81a7-d992eaa31caa None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:57.593174 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c250f-90cf-4c16-81a7-d992eaa31caa None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:57.704962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe2525-a8d8-4502-b5ee-2db9849fcd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:57.706825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe2525-a8d8-4502-b5ee-2db9849fcd27 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:57.707316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe2525-a8d8-4502-b5ee-2db9849fcd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:57.707818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe2525-a8d8-4502-b5ee-2db9849fcd27 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:57.712916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe2525-a8d8-4502-b5ee-2db9849fcd27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:57.718522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe2525-a8d8-4502-b5ee-2db9849fcd27 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:57.904930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1205/2461] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:10:57 2022] POST /identity/v3/users => generated 296 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:10:57.909105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01206c8c-0149-432c-95ad-17aab3e8aec3 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:57.925853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01206c8c-0149-432c-95ad-17aab3e8aec3 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:57.933835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28688232-661b-484b-bdb3-7ec7c01911cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:57.934797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688232-661b-484b-bdb3-7ec7c01911cf admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:57.935299 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688232-661b-484b-bdb3-7ec7c01911cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:57.935691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688232-661b-484b-bdb3-7ec7c01911cf admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:57.936687 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28688232-661b-484b-bdb3-7ec7c01911cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:57.939078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28688232-661b-484b-bdb3-7ec7c01911cf admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:57.958067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1257/2462] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:10:57.973384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe2525-a8d8-4502-b5ee-2db9849fcd27 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:57.985093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe2525-a8d8-4502-b5ee-2db9849fcd27 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:57.990937 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72341806-981b-4afc-9f48-953b155ddf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:57.991645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72341806-981b-4afc-9f48-953b155ddf03 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:57.991959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72341806-981b-4afc-9f48-953b155ddf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:57.992272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72341806-981b-4afc-9f48-953b155ddf03 admin admin] PATH_INFO: `/v3/projects/b146a4d3538043abbe1dc58ddb807cb4/users/13046adb4c8d4129b8ddcc49e9520a4c/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:58.013699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72341806-981b-4afc-9f48-953b155ddf03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b146a4d3538043abbe1dc58ddb807cb4, user_id=13046adb4c8d4129b8ddcc49e9520a4c, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:58.016734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72341806-981b-4afc-9f48-953b155ddf03 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:58.148064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1206/2463] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:10:57 2022] PUT /identity/v3/projects/b146a4d3538043abbe1dc58ddb807cb4/users/13046adb4c8d4129b8ddcc49e9520a4c/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:10:58.154057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28688232-661b-484b-bdb3-7ec7c01911cf admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:58.173453 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28688232-661b-484b-bdb3-7ec7c01911cf admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:58.303907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91e02f-0f31-4c97-a9c4-27b114e5a5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:58.305602 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91e02f-0f31-4c97-a9c4-27b114e5a5ff admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:58.306107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91e02f-0f31-4c97-a9c4-27b114e5a5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:58.306669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91e02f-0f31-4c97-a9c4-27b114e5a5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:58.307942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91e02f-0f31-4c97-a9c4-27b114e5a5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:58.311348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91e02f-0f31-4c97-a9c4-27b114e5a5ff admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:58.332943 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1258/2464] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:10:58.338199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72341806-981b-4afc-9f48-953b155ddf03 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:58.360490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72341806-981b-4afc-9f48-953b155ddf03 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:58.364715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b1885-6055-422c-a023-3e8e9b3022ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:58.366547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b1885-6055-422c-a023-3e8e9b3022ba admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:58.367241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b1885-6055-422c-a023-3e8e9b3022ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:58.367743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b1885-6055-422c-a023-3e8e9b3022ba admin admin] PATH_INFO: `/v3/projects/b146a4d3538043abbe1dc58ddb807cb4/users/13046adb4c8d4129b8ddcc49e9520a4c/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:58.371785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b1885-6055-422c-a023-3e8e9b3022ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b146a4d3538043abbe1dc58ddb807cb4, user_id=13046adb4c8d4129b8ddcc49e9520a4c, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:58.377364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b1885-6055-422c-a023-3e8e9b3022ba admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:58.408520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1207/2465] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:10:58 2022] PUT /identity/v3/projects/b146a4d3538043abbe1dc58ddb807cb4/users/13046adb4c8d4129b8ddcc49e9520a4c/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:10:58.410973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c627288-9cf9-46c4-8748-de14af2bc214 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:58.411488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c627288-9cf9-46c4-8748-de14af2bc214 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:58.411980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c627288-9cf9-46c4-8748-de14af2bc214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:58.468857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-5c627288-9cf9-46c4-8748-de14af2bc214 None None] MFA Rules not processed for user `13046adb4c8d4129b8ddcc49e9520a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:10:58.471163 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5c627288-9cf9-46c4-8748-de14af2bc214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:58.579196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1259/2466] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:10:58 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:10:58.593108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b1885-6055-422c-a023-3e8e9b3022ba admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:58.593979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-904b1885-6055-422c-a023-3e8e9b3022ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:10:58.657118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b1885-6055-422c-a023-3e8e9b3022ba admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:58.660844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e2f16-be30-4581-bd3c-6188255e5915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:58.661522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e2f16-be30-4581-bd3c-6188255e5915 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:58.661806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e2f16-be30-4581-bd3c-6188255e5915 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:58.662088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e2f16-be30-4581-bd3c-6188255e5915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:58.672301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e2f16-be30-4581-bd3c-6188255e5915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:58.673841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e2f16-be30-4581-bd3c-6188255e5915 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:58.689540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1208/2467] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:10:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:10:59.135681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:59.165893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c627288-9cf9-46c4-8748-de14af2bc214 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:10:59.179466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c627288-9cf9-46c4-8748-de14af2bc214 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:10:59.186259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523f6f8-987e-45a1-8250-8089631f614a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:10:59.194018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523f6f8-987e-45a1-8250-8089631f614a service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:10:59.194018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523f6f8-987e-45a1-8250-8089631f614a service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:10:59.194018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523f6f8-987e-45a1-8250-8089631f614a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:10:59.268727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:10:59.281320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523f6f8-987e-45a1-8250-8089631f614a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:10:59.283063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523f6f8-987e-45a1-8250-8089631f614a service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:10:59.295067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1260/2468] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:10:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:11:04.697496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e2f16-be30-4581-bd3c-6188255e5915 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:04.699165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-628e2f16-be30-4581-bd3c-6188255e5915 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:04.769277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e2f16-be30-4581-bd3c-6188255e5915 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:04.773185 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015e89f-34c6-49d9-ab6f-fe6a366e4a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:04.773816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015e89f-34c6-49d9-ab6f-fe6a366e4a08 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:04.774097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015e89f-34c6-49d9-ab6f-fe6a366e4a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:04.774371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015e89f-34c6-49d9-ab6f-fe6a366e4a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:04.776324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f015e89f-34c6-49d9-ab6f-fe6a366e4a08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:04.875192 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015e89f-34c6-49d9-ab6f-fe6a366e4a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:04.877157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015e89f-34c6-49d9-ab6f-fe6a366e4a08 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:04.889820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1209/2469] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:11:10.576137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523f6f8-987e-45a1-8250-8089631f614a service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:10.577700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6523f6f8-987e-45a1-8250-8089631f614a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:10.603819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523f6f8-987e-45a1-8250-8089631f614a service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:10.610711 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:10.611491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:10.611878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:10.612315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] PATH_INFO: `/v3/users/0e8d03fe51534f8e867d0f83c3db2d36` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:10.613702 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8d03fe51534f8e867d0f83c3db2d36)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:10.616888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:10.866128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8d03fe51534f8e867d0f83c3db2d36'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:10.868300 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8d03fe51534f8e867d0f83c3db2d36'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:10.877999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8d03fe51534f8e867d0f83c3db2d36'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:10.879091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8d03fe51534f8e867d0f83c3db2d36'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:10.992686 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a539d618-15a1-4ccb-9a90-0acc5f8918d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8d03fe51534f8e867d0f83c3db2d36'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:11.051010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1261/2470] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:11:10 2022] DELETE /identity/v3/users/0e8d03fe51534f8e867d0f83c3db2d36 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:11:11.604290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015e89f-34c6-49d9-ab6f-fe6a366e4a08 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:11.617762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015e89f-34c6-49d9-ab6f-fe6a366e4a08 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:11.705385 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:11.706281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:11.706638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:11.707220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] PATH_INFO: `/v3/projects/f021c9955236461abc441e2a31421b8b` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:11.708584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f021c9955236461abc441e2a31421b8b)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:11.711631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:11.736313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:11.905794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f021c9955236461abc441e2a31421b8b', 'request_id': 'req-ed12875e-d867-4138-aab7-38deaf7d1cd1'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:11.913318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f021c9955236461abc441e2a31421b8b', 'request_id': 'req-ed12875e-d867-4138-aab7-38deaf7d1cd1'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:11.921339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f021c9955236461abc441e2a31421b8b', 'request_id': 'req-ed12875e-d867-4138-aab7-38deaf7d1cd1'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:12.086647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] The token cache is being invalidate because project f021c9955236461abc441e2a31421b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:11:12.087001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:12.087776 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ed12875e-d867-4138-aab7-38deaf7d1cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:12.089553 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1210/2471] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:11:11 2022] DELETE /identity/v3/projects/f021c9955236461abc441e2a31421b8b => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:11:12.095662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfbb70-9031-43e4-8dae-e1f823f72c35 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:12.096407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfbb70-9031-43e4-8dae-e1f823f72c35 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:12.096407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfbb70-9031-43e4-8dae-e1f823f72c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:12.119737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-11dfbb70-9031-43e4-8dae-e1f823f72c35 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:12.121536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-11dfbb70-9031-43e4-8dae-e1f823f72c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:12.206886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1262/2472] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:12.216760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045370ab-188f-4414-9112-a25c9286a749 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:12.217189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045370ab-188f-4414-9112-a25c9286a749 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:12.217638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045370ab-188f-4414-9112-a25c9286a749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:12.241941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-045370ab-188f-4414-9112-a25c9286a749 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:12.243223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-045370ab-188f-4414-9112-a25c9286a749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:12.252688 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1211/2473] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:11:12.258043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfbb70-9031-43e4-8dae-e1f823f72c35 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:12.280053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfbb70-9031-43e4-8dae-e1f823f72c35 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:12.289426 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a368e-bcba-4c55-83ee-1c78d41752d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:12.297456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a368e-bcba-4c55-83ee-1c78d41752d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:12.297456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a368e-bcba-4c55-83ee-1c78d41752d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:12.297456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a368e-bcba-4c55-83ee-1c78d41752d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:12.297456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a368e-bcba-4c55-83ee-1c78d41752d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:11:12.297456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a368e-bcba-4c55-83ee-1c78d41752d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:12.297456 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a368e-bcba-4c55-83ee-1c78d41752d6 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:12.322935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1263/2474] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:11:12.328880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045370ab-188f-4414-9112-a25c9286a749 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:12.342113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045370ab-188f-4414-9112-a25c9286a749 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:12.348106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66623a85-42a0-43ca-beb6-b1ad29bd834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:12.348858 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66623a85-42a0-43ca-beb6-b1ad29bd834a admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:12.349231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66623a85-42a0-43ca-beb6-b1ad29bd834a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:12.349607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66623a85-42a0-43ca-beb6-b1ad29bd834a admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:12.350333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66623a85-42a0-43ca-beb6-b1ad29bd834a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:12.352591 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66623a85-42a0-43ca-beb6-b1ad29bd834a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:12.834159 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1212/2475] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:11:12 2022] POST /identity/v3/projects => generated 383 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:11:12.835953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a368e-bcba-4c55-83ee-1c78d41752d6 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:12.858364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a368e-bcba-4c55-83ee-1c78d41752d6 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:12.945184 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:12.967225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82357178-0b63-4e0b-a661-cb592f471009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:12.968605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82357178-0b63-4e0b-a661-cb592f471009 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:12.969110 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82357178-0b63-4e0b-a661-cb592f471009 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:12.969622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82357178-0b63-4e0b-a661-cb592f471009 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:12.970841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82357178-0b63-4e0b-a661-cb592f471009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:12.974851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82357178-0b63-4e0b-a661-cb592f471009 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:13.202557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1264/2476] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:11:12 2022] POST /identity/v3/users => generated 292 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:11:13.206936 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66623a85-42a0-43ca-beb6-b1ad29bd834a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:13.220366 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66623a85-42a0-43ca-beb6-b1ad29bd834a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:13.227897 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce40c6-e619-4ee1-a7c6-f939e27bcc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:13.228673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce40c6-e619-4ee1-a7c6-f939e27bcc9b admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:13.229376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce40c6-e619-4ee1-a7c6-f939e27bcc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:13.229376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce40c6-e619-4ee1-a7c6-f939e27bcc9b admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:13.229878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce40c6-e619-4ee1-a7c6-f939e27bcc9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:13.231576 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce40c6-e619-4ee1-a7c6-f939e27bcc9b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:13.243414 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1213/2477] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:11:13.250650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82357178-0b63-4e0b-a661-cb592f471009 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:13.264263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82357178-0b63-4e0b-a661-cb592f471009 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:13.269323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d1c82-496d-4843-9781-c5da6da2b685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:13.269955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d1c82-496d-4843-9781-c5da6da2b685 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:13.270238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d1c82-496d-4843-9781-c5da6da2b685 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:13.270529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d1c82-496d-4843-9781-c5da6da2b685 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:13.271196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d1c82-496d-4843-9781-c5da6da2b685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:13.273074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d1c82-496d-4843-9781-c5da6da2b685 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:13.287223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1265/2478] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:11:13.292066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce40c6-e619-4ee1-a7c6-f939e27bcc9b admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:13.303087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce40c6-e619-4ee1-a7c6-f939e27bcc9b admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:13.309259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef6672-fcd1-4bfa-b29e-acce86502ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:13.310080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef6672-fcd1-4bfa-b29e-acce86502ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:13.310502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef6672-fcd1-4bfa-b29e-acce86502ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:13.310937 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef6672-fcd1-4bfa-b29e-acce86502ab1 admin admin] PATH_INFO: `/v3/projects/63435c4e170a4cd19a39997da716d345/users/fa19a37a4ae84e1487963e1dea311eb7/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:13.330505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef6672-fcd1-4bfa-b29e-acce86502ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63435c4e170a4cd19a39997da716d345, user_id=fa19a37a4ae84e1487963e1dea311eb7, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:13.333584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef6672-fcd1-4bfa-b29e-acce86502ab1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:13.542381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1214/2479] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:11:13 2022] PUT /identity/v3/projects/63435c4e170a4cd19a39997da716d345/users/fa19a37a4ae84e1487963e1dea311eb7/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:13.555940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb03a8-de3b-4173-af0b-6bcbf480d302 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:13.556499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb03a8-de3b-4173-af0b-6bcbf480d302 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:13.556990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb03a8-de3b-4173-af0b-6bcbf480d302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:13.613442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2ecb03a8-de3b-4173-af0b-6bcbf480d302 None None] MFA Rules not processed for user `fa19a37a4ae84e1487963e1dea311eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:13.615680 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2ecb03a8-de3b-4173-af0b-6bcbf480d302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:13.706645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1266/2480] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:11:13.795685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef6672-fcd1-4bfa-b29e-acce86502ab1 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:13.798930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1aef6672-fcd1-4bfa-b29e-acce86502ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:13.909848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef6672-fcd1-4bfa-b29e-acce86502ab1 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:13.921049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a6821-c661-4475-bfff-e5e638efef49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:13.922187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a6821-c661-4475-bfff-e5e638efef49 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:13.922842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a6821-c661-4475-bfff-e5e638efef49 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:13.923212 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a6821-c661-4475-bfff-e5e638efef49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:13.942657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a6821-c661-4475-bfff-e5e638efef49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:13.945726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a6821-c661-4475-bfff-e5e638efef49 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:13.966951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1215/2481] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:11:14.339845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb03a8-de3b-4173-af0b-6bcbf480d302 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:14.341538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2ecb03a8-de3b-4173-af0b-6bcbf480d302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:14.424873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb03a8-de3b-4173-af0b-6bcbf480d302 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:14.431024 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:14.431690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:14.431975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:14.432312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] PATH_INFO: `/v3/users/a11dd1e91e024801a91a1f456fdf68db` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:14.433617 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11dd1e91e024801a91a1f456fdf68db)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:14.435962 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:14.751934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11dd1e91e024801a91a1f456fdf68db'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:14.753088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11dd1e91e024801a91a1f456fdf68db'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:14.767401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11dd1e91e024801a91a1f456fdf68db'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:14.768695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11dd1e91e024801a91a1f456fdf68db'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:14.773655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:14.935621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11dd1e91e024801a91a1f456fdf68db'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:14.987234 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1267/2482] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:11:14 2022] DELETE /identity/v3/users/a11dd1e91e024801a91a1f456fdf68db => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:14.993940 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894f25b-4e37-4518-9203-6e7169246d64 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:14.994790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894f25b-4e37-4518-9203-6e7169246d64 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:14.995086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894f25b-4e37-4518-9203-6e7169246d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:15.031845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d894f25b-4e37-4518-9203-6e7169246d64 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:15.033497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d894f25b-4e37-4518-9203-6e7169246d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:15.086823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:15.214299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1216/2483] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:15.217064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:15.226487 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:15.305925 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e3149-08fc-461d-8542-5b65e14ef008 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:15.312740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7d2b9-42fa-4e99-87f8-60516893b031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:15.313730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7d2b9-42fa-4e99-87f8-60516893b031 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:15.314044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7d2b9-42fa-4e99-87f8-60516893b031 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:15.314325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7d2b9-42fa-4e99-87f8-60516893b031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:15.330524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d7d2b9-42fa-4e99-87f8-60516893b031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:15.330524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d7d2b9-42fa-4e99-87f8-60516893b031 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:15.360519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1268/2484] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:11:16.025888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894f25b-4e37-4518-9203-6e7169246d64 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:16.028629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d894f25b-4e37-4518-9203-6e7169246d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:16.049792 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894f25b-4e37-4518-9203-6e7169246d64 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:16.056595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:16.057726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:16.058160 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:16.058747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] PATH_INFO: `/v3/projects/90111ff0ed744a86aed1ed3c59f53dfe` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:16.061348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=90111ff0ed744a86aed1ed3c59f53dfe)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:16.065129 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:16.350063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90111ff0ed744a86aed1ed3c59f53dfe', 'request_id': 'req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:16.356780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90111ff0ed744a86aed1ed3c59f53dfe', 'request_id': 'req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:16.364707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90111ff0ed744a86aed1ed3c59f53dfe', 'request_id': 'req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:16.581659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] The token cache is being invalidate because project 90111ff0ed744a86aed1ed3c59f53dfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:11:16.581659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:16.582435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:16.584509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1217/2485] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:11:16 2022] DELETE /identity/v3/projects/90111ff0ed744a86aed1ed3c59f53dfe => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:11:16.591721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7d2b9-42fa-4e99-87f8-60516893b031 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:16.607876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7d2b9-42fa-4e99-87f8-60516893b031 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:16.719677 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:16.720581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:16.721023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:16.721483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] PATH_INFO: `/v3/users/13046adb4c8d4129b8ddcc49e9520a4c` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:16.723192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13046adb4c8d4129b8ddcc49e9520a4c)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:16.725524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:17.053312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13046adb4c8d4129b8ddcc49e9520a4c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:17.054201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13046adb4c8d4129b8ddcc49e9520a4c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:17.063739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13046adb4c8d4129b8ddcc49e9520a4c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:17.064826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13046adb4c8d4129b8ddcc49e9520a4c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:17.177598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0a809bf4-1a1b-4228-9092-33755f7b1bb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13046adb4c8d4129b8ddcc49e9520a4c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:17.195896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:17.202323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1269/2486] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:11:16 2022] DELETE /identity/v3/users/13046adb4c8d4129b8ddcc49e9520a4c => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:17.858246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:17.861565 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:17.880366 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddaf3e-fab9-49fc-84ea-56dc5e366c7c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:17.964077 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:17.965192 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:17.965610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:17.966031 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] PATH_INFO: `/v3/projects/b146a4d3538043abbe1dc58ddb807cb4` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:17.967930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b146a4d3538043abbe1dc58ddb807cb4)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:17.970658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:18.192014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b146a4d3538043abbe1dc58ddb807cb4', 'request_id': 'req-28ef001e-151a-4844-8ed1-6bb679dbb23c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:18.193502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:18.201221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b146a4d3538043abbe1dc58ddb807cb4', 'request_id': 'req-28ef001e-151a-4844-8ed1-6bb679dbb23c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:18.209658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b146a4d3538043abbe1dc58ddb807cb4', 'request_id': 'req-28ef001e-151a-4844-8ed1-6bb679dbb23c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:18.418743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] The token cache is being invalidate because project b146a4d3538043abbe1dc58ddb807cb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:11:18.419173 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:18.420009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-28ef001e-151a-4844-8ed1-6bb679dbb23c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:18.422377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1218/2487] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:11:17 2022] DELETE /identity/v3/projects/b146a4d3538043abbe1dc58ddb807cb4 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:18.429522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae72fd3-3a97-4103-a057-27d864b556bc None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:18.430176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae72fd3-3a97-4103-a057-27d864b556bc None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:18.430176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae72fd3-3a97-4103-a057-27d864b556bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:18.455409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-cae72fd3-3a97-4103-a057-27d864b556bc None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:18.456663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cae72fd3-3a97-4103-a057-27d864b556bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:18.580274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1270/2488] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:18.588721 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57568e-11ec-4e00-aba8-482dace5d91a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:18.589589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57568e-11ec-4e00-aba8-482dace5d91a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:18.590548 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57568e-11ec-4e00-aba8-482dace5d91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:18.612980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-cc57568e-11ec-4e00-aba8-482dace5d91a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:18.614377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cc57568e-11ec-4e00-aba8-482dace5d91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:18.623275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1219/2489] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:18.628574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae72fd3-3a97-4103-a057-27d864b556bc None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:18.641553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae72fd3-3a97-4103-a057-27d864b556bc None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:18.648229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ef68b-9bd5-4086-ade0-3ca158007ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:18.652223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef68b-9bd5-4086-ade0-3ca158007ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:18.652223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef68b-9bd5-4086-ade0-3ca158007ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:18.652223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef68b-9bd5-4086-ade0-3ca158007ca5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:18.652223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ef68b-9bd5-4086-ade0-3ca158007ca5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:11:18.652223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ef68b-9bd5-4086-ade0-3ca158007ca5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:18.653011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ef68b-9bd5-4086-ade0-3ca158007ca5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:18.672388 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1271/2490] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:11:18.674088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57568e-11ec-4e00-aba8-482dace5d91a None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:18.685636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57568e-11ec-4e00-aba8-482dace5d91a None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:18.692660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df661e02-d92b-4de1-8a63-0acad86dcbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:18.693603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df661e02-d92b-4de1-8a63-0acad86dcbdd admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:18.693995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df661e02-d92b-4de1-8a63-0acad86dcbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:18.694543 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df661e02-d92b-4de1-8a63-0acad86dcbdd admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:18.695492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df661e02-d92b-4de1-8a63-0acad86dcbdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:18.697703 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df661e02-d92b-4de1-8a63-0acad86dcbdd admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:18.843210 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1220/2491] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:11:18 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:11:18.848530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ef68b-9bd5-4086-ade0-3ca158007ca5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:18.862959 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ef68b-9bd5-4086-ade0-3ca158007ca5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:18.970733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a9422-6dd9-40bb-bed9-1e6dc711e998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:18.971506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a9422-6dd9-40bb-bed9-1e6dc711e998 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:18.971854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a9422-6dd9-40bb-bed9-1e6dc711e998 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:18.972242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a9422-6dd9-40bb-bed9-1e6dc711e998 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:18.972952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a9422-6dd9-40bb-bed9-1e6dc711e998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:18.975385 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a9422-6dd9-40bb-bed9-1e6dc711e998 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:19.086857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1272/2492] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:11:18 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:11:19.092664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df661e02-d92b-4de1-8a63-0acad86dcbdd admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:19.111426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df661e02-d92b-4de1-8a63-0acad86dcbdd admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:19.119409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c4b53-fcbf-4742-861e-8f2ca38b2904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:19.120592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c4b53-fcbf-4742-861e-8f2ca38b2904 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:19.121124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c4b53-fcbf-4742-861e-8f2ca38b2904 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:19.121556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c4b53-fcbf-4742-861e-8f2ca38b2904 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:19.122439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c4b53-fcbf-4742-861e-8f2ca38b2904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:19.124401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c4b53-fcbf-4742-861e-8f2ca38b2904 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:19.137691 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1221/2493] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:11:19.142761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a9422-6dd9-40bb-bed9-1e6dc711e998 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:19.162477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a9422-6dd9-40bb-bed9-1e6dc711e998 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:19.171254 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be6a0e-877f-4f15-9a1a-ab64e671ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:19.172375 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be6a0e-877f-4f15-9a1a-ab64e671ed16 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:19.172916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be6a0e-877f-4f15-9a1a-ab64e671ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:19.173397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be6a0e-877f-4f15-9a1a-ab64e671ed16 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:19.174646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be6a0e-877f-4f15-9a1a-ab64e671ed16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:19.177926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be6a0e-877f-4f15-9a1a-ab64e671ed16 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:19.201635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1273/2494] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:11:19.213239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c4b53-fcbf-4742-861e-8f2ca38b2904 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:19.225034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c4b53-fcbf-4742-861e-8f2ca38b2904 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:19.230569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b4901-bcb4-44ac-91e5-abbec5518ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:19.231456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4901-bcb4-44ac-91e5-abbec5518ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:19.231732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4901-bcb4-44ac-91e5-abbec5518ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:19.232003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4901-bcb4-44ac-91e5-abbec5518ea2 admin admin] PATH_INFO: `/v3/projects/ef21a298b8374520aad4854e6dd343db/users/c91a2dbf9e24494883186f0e2f6ff4d2/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:19.250829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b4901-bcb4-44ac-91e5-abbec5518ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef21a298b8374520aad4854e6dd343db, user_id=c91a2dbf9e24494883186f0e2f6ff4d2, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:19.254093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b4901-bcb4-44ac-91e5-abbec5518ea2 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:19.442828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1222/2495] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:11:19 2022] PUT /identity/v3/projects/ef21a298b8374520aad4854e6dd343db/users/c91a2dbf9e24494883186f0e2f6ff4d2/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:11:19.456070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23508a6-9644-4527-87fe-d57331db709f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:19.457090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23508a6-9644-4527-87fe-d57331db709f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:19.457319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23508a6-9644-4527-87fe-d57331db709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:19.494051 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d23508a6-9644-4527-87fe-d57331db709f None None] MFA Rules not processed for user `c91a2dbf9e24494883186f0e2f6ff4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:19.496846 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d23508a6-9644-4527-87fe-d57331db709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:19.594873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:19.619646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1274/2496] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:19.621905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b4901-bcb4-44ac-91e5-abbec5518ea2 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:19.726055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b4901-bcb4-44ac-91e5-abbec5518ea2 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:19.732630 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a8452-6b2d-4bc5-a30f-3ab57a1890eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:19.733333 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a8452-6b2d-4bc5-a30f-3ab57a1890eb admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:19.733670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a8452-6b2d-4bc5-a30f-3ab57a1890eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:19.734071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a8452-6b2d-4bc5-a30f-3ab57a1890eb admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:19.734854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a8452-6b2d-4bc5-a30f-3ab57a1890eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:19.737591 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a8452-6b2d-4bc5-a30f-3ab57a1890eb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:19.888959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1223/2497] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:11:19 2022] POST /identity/v3/projects => generated 375 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:11:19.894508 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23508a6-9644-4527-87fe-d57331db709f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:19.914529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23508a6-9644-4527-87fe-d57331db709f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:20.014582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b8ff5-e45d-4e74-a9e0-b72eed6488f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:20.015756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b8ff5-e45d-4e74-a9e0-b72eed6488f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:20.016364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b8ff5-e45d-4e74-a9e0-b72eed6488f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:20.016931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b8ff5-e45d-4e74-a9e0-b72eed6488f8 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:20.018117 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b8ff5-e45d-4e74-a9e0-b72eed6488f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:20.022018 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b8ff5-e45d-4e74-a9e0-b72eed6488f8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:20.183858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1275/2498] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:11:19 2022] POST /identity/v3/users => generated 294 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:11:20.189025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a8452-6b2d-4bc5-a30f-3ab57a1890eb admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:20.208538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a8452-6b2d-4bc5-a30f-3ab57a1890eb admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:20.216994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cb5a1-3a1d-4a9c-abed-9f17ca17895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:20.217803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cb5a1-3a1d-4a9c-abed-9f17ca17895b admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:20.218102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cb5a1-3a1d-4a9c-abed-9f17ca17895b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:20.218562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cb5a1-3a1d-4a9c-abed-9f17ca17895b admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:20.219717 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cb5a1-3a1d-4a9c-abed-9f17ca17895b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:20.223492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cb5a1-3a1d-4a9c-abed-9f17ca17895b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:20.245261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1224/2499] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:11:20.252376 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b8ff5-e45d-4e74-a9e0-b72eed6488f8 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:20.266568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b8ff5-e45d-4e74-a9e0-b72eed6488f8 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:20.272437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7c28d-985b-4df2-bdb2-a86e3e8e1600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:20.273175 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7c28d-985b-4df2-bdb2-a86e3e8e1600 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:20.273499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7c28d-985b-4df2-bdb2-a86e3e8e1600 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:20.273825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7c28d-985b-4df2-bdb2-a86e3e8e1600 admin admin] PATH_INFO: `/v3/projects/8eb1d0e07fc64aef8209fd24c5d27d6f/users/b86dc78b4691490393ae852df86f070b/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:20.294401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7c28d-985b-4df2-bdb2-a86e3e8e1600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb1d0e07fc64aef8209fd24c5d27d6f, user_id=b86dc78b4691490393ae852df86f070b, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:20.298125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7c28d-985b-4df2-bdb2-a86e3e8e1600 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:20.405552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1276/2500] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:11:20 2022] PUT /identity/v3/projects/8eb1d0e07fc64aef8209fd24c5d27d6f/users/b86dc78b4691490393ae852df86f070b/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:11:20.410237 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cb5a1-3a1d-4a9c-abed-9f17ca17895b admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:20.422033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cb5a1-3a1d-4a9c-abed-9f17ca17895b admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:20.486962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:20.501788 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d162a4-c7bd-4b23-8868-e66844e33726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:20.506891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d162a4-c7bd-4b23-8868-e66844e33726 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:20.506891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d162a4-c7bd-4b23-8868-e66844e33726 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:20.506891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d162a4-c7bd-4b23-8868-e66844e33726 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:20.506891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d162a4-c7bd-4b23-8868-e66844e33726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:20.506891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d162a4-c7bd-4b23-8868-e66844e33726 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:20.520895 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1225/2501] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:11:20.525226 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7c28d-985b-4df2-bdb2-a86e3e8e1600 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:20.543910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7c28d-985b-4df2-bdb2-a86e3e8e1600 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:20.551437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba69d9-0c52-4c7b-953a-80105ad71b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:20.552558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba69d9-0c52-4c7b-953a-80105ad71b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:20.555343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba69d9-0c52-4c7b-953a-80105ad71b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:20.555343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba69d9-0c52-4c7b-953a-80105ad71b04 admin admin] PATH_INFO: `/v3/projects/8eb1d0e07fc64aef8209fd24c5d27d6f/users/b86dc78b4691490393ae852df86f070b/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:20.556981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba69d9-0c52-4c7b-953a-80105ad71b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb1d0e07fc64aef8209fd24c5d27d6f, user_id=b86dc78b4691490393ae852df86f070b, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:20.561782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba69d9-0c52-4c7b-953a-80105ad71b04 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:20.582743 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1277/2502] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:11:20 2022] PUT /identity/v3/projects/8eb1d0e07fc64aef8209fd24c5d27d6f/users/b86dc78b4691490393ae852df86f070b/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:20.602736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8a098-da1d-4665-b925-d5b5adc9bbae None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:20.603356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8a098-da1d-4665-b925-d5b5adc9bbae None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:20.603898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8a098-da1d-4665-b925-d5b5adc9bbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:20.651016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d0f8a098-da1d-4665-b925-d5b5adc9bbae None None] MFA Rules not processed for user `b86dc78b4691490393ae852df86f070b`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:20.653278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d0f8a098-da1d-4665-b925-d5b5adc9bbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:20.749795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1226/2503] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:20.762066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba69d9-0c52-4c7b-953a-80105ad71b04 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:20.763908 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7dba69d9-0c52-4c7b-953a-80105ad71b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:20.831107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba69d9-0c52-4c7b-953a-80105ad71b04 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:20.838631 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa9863-1a30-4e88-8bf0-277f122eb45f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:20.840279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa9863-1a30-4e88-8bf0-277f122eb45f service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:20.840751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa9863-1a30-4e88-8bf0-277f122eb45f service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:20.841210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa9863-1a30-4e88-8bf0-277f122eb45f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:20.907227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa9863-1a30-4e88-8bf0-277f122eb45f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:20.908911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa9863-1a30-4e88-8bf0-277f122eb45f service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:20.915311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:20.920972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1278/2504] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:11:25.198593 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8a098-da1d-4665-b925-d5b5adc9bbae None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:25.198593 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d0f8a098-da1d-4665-b925-d5b5adc9bbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:25.275624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8a098-da1d-4665-b925-d5b5adc9bbae None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:25.281151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:25.281879 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:25.282155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:25.282428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] PATH_INFO: `/v3/users/fa19a37a4ae84e1487963e1dea311eb7` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:25.283598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa19a37a4ae84e1487963e1dea311eb7)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:25.285584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:25.707580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa19a37a4ae84e1487963e1dea311eb7'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:25.715828 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa19a37a4ae84e1487963e1dea311eb7'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:25.815215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa19a37a4ae84e1487963e1dea311eb7'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:25.919496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa19a37a4ae84e1487963e1dea311eb7'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:25.928119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:25.934412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa19a37a4ae84e1487963e1dea311eb7'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:25.940532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1227/2505] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:11:25 2022] DELETE /identity/v3/users/fa19a37a4ae84e1487963e1dea311eb7 => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:25.945519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42825990-dd1b-48e1-ad7f-1d95cbe8cb1a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:25.946453 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42825990-dd1b-48e1-ad7f-1d95cbe8cb1a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:25.946453 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42825990-dd1b-48e1-ad7f-1d95cbe8cb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:25.970434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-42825990-dd1b-48e1-ad7f-1d95cbe8cb1a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:25.971840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-42825990-dd1b-48e1-ad7f-1d95cbe8cb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:26.037260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1279/2506] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:26.048429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:26.049992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:26.115458 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420cc10-ea16-4ad7-8b27-433062aeab64 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:26.119066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd0a55-78d7-440c-9329-69bd3e5d4e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:26.119670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd0a55-78d7-440c-9329-69bd3e5d4e57 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:26.120776 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd0a55-78d7-440c-9329-69bd3e5d4e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:26.121250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd0a55-78d7-440c-9329-69bd3e5d4e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:26.131746 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd0a55-78d7-440c-9329-69bd3e5d4e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:26.133232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd0a55-78d7-440c-9329-69bd3e5d4e57 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:26.144624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1228/2507] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:11:26.775265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42825990-dd1b-48e1-ad7f-1d95cbe8cb1a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:26.775265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-42825990-dd1b-48e1-ad7f-1d95cbe8cb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:26.795330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42825990-dd1b-48e1-ad7f-1d95cbe8cb1a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:26.805335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:26.806027 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:26.806411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:26.806696 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] PATH_INFO: `/v3/projects/63435c4e170a4cd19a39997da716d345` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:26.808320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63435c4e170a4cd19a39997da716d345)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:26.810670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:27.069351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63435c4e170a4cd19a39997da716d345', 'request_id': 'req-6cb9094b-c44d-4ca1-92f0-59dece634782'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:27.076203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63435c4e170a4cd19a39997da716d345', 'request_id': 'req-6cb9094b-c44d-4ca1-92f0-59dece634782'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:27.168333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63435c4e170a4cd19a39997da716d345', 'request_id': 'req-6cb9094b-c44d-4ca1-92f0-59dece634782'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:27.181323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] The token cache is being invalidate because project 63435c4e170a4cd19a39997da716d345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:11:27.181636 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:27.182870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6cb9094b-c44d-4ca1-92f0-59dece634782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:27.184542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1280/2508] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:11:26 2022] DELETE /identity/v3/projects/63435c4e170a4cd19a39997da716d345 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:27.190171 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bba5a3-508f-4503-b5bf-49b78b6b1d30 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:27.190977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bba5a3-508f-4503-b5bf-49b78b6b1d30 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:27.190977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bba5a3-508f-4503-b5bf-49b78b6b1d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:27.213429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e6bba5a3-508f-4503-b5bf-49b78b6b1d30 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:27.214702 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e6bba5a3-508f-4503-b5bf-49b78b6b1d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:27.283963 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1229/2509] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:27.292953 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffe3ad-4505-4d38-9aa5-0af567f2cee5 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:27.293221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffe3ad-4505-4d38-9aa5-0af567f2cee5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:27.293543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffe3ad-4505-4d38-9aa5-0af567f2cee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:27.316214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e5ffe3ad-4505-4d38-9aa5-0af567f2cee5 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:27.317542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e5ffe3ad-4505-4d38-9aa5-0af567f2cee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:27.327160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1281/2510] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:27.331611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bba5a3-508f-4503-b5bf-49b78b6b1d30 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:27.342594 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bba5a3-508f-4503-b5bf-49b78b6b1d30 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:27.347446 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a4e5a-52f2-4958-871e-630809308718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:27.348051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a4e5a-52f2-4958-871e-630809308718 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:27.348364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a4e5a-52f2-4958-871e-630809308718 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:27.348640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a4e5a-52f2-4958-871e-630809308718 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:27.349325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a4e5a-52f2-4958-871e-630809308718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:11:27.349648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a4e5a-52f2-4958-871e-630809308718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:27.351388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a4e5a-52f2-4958-871e-630809308718 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:27.365005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1230/2511] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:11:27.369599 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffe3ad-4505-4d38-9aa5-0af567f2cee5 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:27.380151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffe3ad-4505-4d38-9aa5-0af567f2cee5 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:27.385019 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27ebf7-2a21-4bc2-9e77-90250707956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:27.385681 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27ebf7-2a21-4bc2-9e77-90250707956f admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:27.385960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27ebf7-2a21-4bc2-9e77-90250707956f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:27.386233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27ebf7-2a21-4bc2-9e77-90250707956f admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:27.386992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27ebf7-2a21-4bc2-9e77-90250707956f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:27.388941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27ebf7-2a21-4bc2-9e77-90250707956f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:27.530261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1282/2512] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:11:27 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:11:27.534647 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a4e5a-52f2-4958-871e-630809308718 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:27.546029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a4e5a-52f2-4958-871e-630809308718 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:27.613925 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661848d-8e97-4131-b081-3fc091bd63eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:27.614592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661848d-8e97-4131-b081-3fc091bd63eb admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:27.614899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661848d-8e97-4131-b081-3fc091bd63eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:27.615362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661848d-8e97-4131-b081-3fc091bd63eb admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:27.615930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661848d-8e97-4131-b081-3fc091bd63eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:27.618327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661848d-8e97-4131-b081-3fc091bd63eb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:27.769544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1231/2513] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:11:27 2022] POST /identity/v3/users => generated 283 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:11:27.774386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27ebf7-2a21-4bc2-9e77-90250707956f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:27.787088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27ebf7-2a21-4bc2-9e77-90250707956f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:27.795377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c90c59-aeb9-4303-b258-da812032412e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:27.802461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c90c59-aeb9-4303-b258-da812032412e admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:27.802461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c90c59-aeb9-4303-b258-da812032412e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:27.802461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c90c59-aeb9-4303-b258-da812032412e admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:27.802461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c90c59-aeb9-4303-b258-da812032412e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:27.802461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c90c59-aeb9-4303-b258-da812032412e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:27.812021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1283/2514] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:11:27.817162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661848d-8e97-4131-b081-3fc091bd63eb admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:27.835231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661848d-8e97-4131-b081-3fc091bd63eb admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:27.843600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a5a60-f1d3-4767-a28d-9add02f772e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:27.844544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a5a60-f1d3-4767-a28d-9add02f772e3 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:27.844833 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a5a60-f1d3-4767-a28d-9add02f772e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:27.845136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a5a60-f1d3-4767-a28d-9add02f772e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:27.845829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a5a60-f1d3-4767-a28d-9add02f772e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:27.848880 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a5a60-f1d3-4767-a28d-9add02f772e3 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:27.852922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:27.869066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1232/2515] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:11:27.874337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c90c59-aeb9-4303-b258-da812032412e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:27.886893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c90c59-aeb9-4303-b258-da812032412e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:27.892194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d4b2a-a05e-4b6a-b589-972df8b096f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:27.892896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d4b2a-a05e-4b6a-b589-972df8b096f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:27.893194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d4b2a-a05e-4b6a-b589-972df8b096f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:27.893470 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d4b2a-a05e-4b6a-b589-972df8b096f2 admin admin] PATH_INFO: `/v3/projects/e2d28572de504e558e7d96f854fcda9e/users/2f57112ad4754f09add6a84d588bf873/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:27.911441 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d4b2a-a05e-4b6a-b589-972df8b096f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d28572de504e558e7d96f854fcda9e, user_id=2f57112ad4754f09add6a84d588bf873, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:27.914255 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d4b2a-a05e-4b6a-b589-972df8b096f2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:28.057989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1284/2516] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:11:27 2022] PUT /identity/v3/projects/e2d28572de504e558e7d96f854fcda9e/users/2f57112ad4754f09add6a84d588bf873/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:11:28.068870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94402b65-bbc8-4f3a-adf5-38468c66abcf None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:28.069208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94402b65-bbc8-4f3a-adf5-38468c66abcf None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:28.069544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94402b65-bbc8-4f3a-adf5-38468c66abcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:28.103597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-94402b65-bbc8-4f3a-adf5-38468c66abcf None None] MFA Rules not processed for user `2f57112ad4754f09add6a84d588bf873`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:28.104947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-94402b65-bbc8-4f3a-adf5-38468c66abcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:28.119512 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d4b2a-a05e-4b6a-b589-972df8b096f2 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:28.121551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ad7d4b2a-a05e-4b6a-b589-972df8b096f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:28.150166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:28.181239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1233/2517] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:28.186580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94402b65-bbc8-4f3a-adf5-38468c66abcf None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:28.192109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d4b2a-a05e-4b6a-b589-972df8b096f2 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:28.195870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8badc-52c7-4ad6-b4b0-2f72faabe72e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:28.196543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8badc-52c7-4ad6-b4b0-2f72faabe72e service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:28.196823 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8badc-52c7-4ad6-b4b0-2f72faabe72e service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:28.197097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8badc-52c7-4ad6-b4b0-2f72faabe72e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:28.199489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-25d8badc-52c7-4ad6-b4b0-2f72faabe72e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:28.255943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94402b65-bbc8-4f3a-adf5-38468c66abcf None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:28.262032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745508ed-df7a-41f0-9c61-9f84f6e6a3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:28.262972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745508ed-df7a-41f0-9c61-9f84f6e6a3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:28.263192 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745508ed-df7a-41f0-9c61-9f84f6e6a3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:28.263437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745508ed-df7a-41f0-9c61-9f84f6e6a3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:28.268390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745508ed-df7a-41f0-9c61-9f84f6e6a3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:28.268390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745508ed-df7a-41f0-9c61-9f84f6e6a3e0 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:28.270737 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8badc-52c7-4ad6-b4b0-2f72faabe72e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:28.272242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8badc-52c7-4ad6-b4b0-2f72faabe72e service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:28.284305 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1285/2518] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:11:28.639377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1234/2519] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:11:28 2022] POST /identity/v3/projects => generated 365 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:11:28.660052 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8badc-52c7-4ad6-b4b0-2f72faabe72e service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:28.671362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8badc-52c7-4ad6-b4b0-2f72faabe72e service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:28.735526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c03f7-d646-4ba5-bb4f-a527e96591b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:28.736232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c03f7-d646-4ba5-bb4f-a527e96591b7 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:28.736522 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c03f7-d646-4ba5-bb4f-a527e96591b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:28.736797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c03f7-d646-4ba5-bb4f-a527e96591b7 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:28.737476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c03f7-d646-4ba5-bb4f-a527e96591b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:28.739464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c03f7-d646-4ba5-bb4f-a527e96591b7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:28.899138 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1286/2520] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:11:28 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:11:28.900401 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745508ed-df7a-41f0-9c61-9f84f6e6a3e0 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:28.911686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745508ed-df7a-41f0-9c61-9f84f6e6a3e0 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:28.917171 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605339c4-e840-4ed7-b9f4-dd6853d2c554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:28.917802 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605339c4-e840-4ed7-b9f4-dd6853d2c554 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:28.918099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605339c4-e840-4ed7-b9f4-dd6853d2c554 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:28.918379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605339c4-e840-4ed7-b9f4-dd6853d2c554 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:28.918975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605339c4-e840-4ed7-b9f4-dd6853d2c554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:28.920652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605339c4-e840-4ed7-b9f4-dd6853d2c554 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:28.932747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1235/2521] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:11:28.937647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c03f7-d646-4ba5-bb4f-a527e96591b7 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:28.948620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c03f7-d646-4ba5-bb4f-a527e96591b7 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:28.953594 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3188f04-1794-47f5-b5a3-667572c3629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:28.954296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3188f04-1794-47f5-b5a3-667572c3629c admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:28.954584 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3188f04-1794-47f5-b5a3-667572c3629c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:28.956296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3188f04-1794-47f5-b5a3-667572c3629c admin admin] PATH_INFO: `/v3/projects/8f01b673cff84a7995346e849e05662d/users/83327ce96ae348689492e4159b7cbe04/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:28.972515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3188f04-1794-47f5-b5a3-667572c3629c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f01b673cff84a7995346e849e05662d, user_id=83327ce96ae348689492e4159b7cbe04, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:28.975202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3188f04-1794-47f5-b5a3-667572c3629c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:29.086643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1287/2522] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:11:28 2022] PUT /identity/v3/projects/8f01b673cff84a7995346e849e05662d/users/83327ce96ae348689492e4159b7cbe04/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:11:29.091229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605339c4-e840-4ed7-b9f4-dd6853d2c554 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:29.107120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605339c4-e840-4ed7-b9f4-dd6853d2c554 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:29.183870 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960f5ca-fc76-438c-a77e-cc824418cbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:29.184732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960f5ca-fc76-438c-a77e-cc824418cbeb admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:29.185355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960f5ca-fc76-438c-a77e-cc824418cbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:29.185916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960f5ca-fc76-438c-a77e-cc824418cbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:29.187246 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960f5ca-fc76-438c-a77e-cc824418cbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:29.190403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960f5ca-fc76-438c-a77e-cc824418cbeb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:29.195018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:29.214169 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1236/2523] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:11:29.220644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3188f04-1794-47f5-b5a3-667572c3629c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:29.233165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3188f04-1794-47f5-b5a3-667572c3629c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:29.238499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492e28e-2b18-4ca8-9e70-990eac855a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:29.239264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492e28e-2b18-4ca8-9e70-990eac855a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:29.239549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492e28e-2b18-4ca8-9e70-990eac855a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:29.239829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492e28e-2b18-4ca8-9e70-990eac855a39 admin admin] PATH_INFO: `/v3/projects/8f01b673cff84a7995346e849e05662d/users/83327ce96ae348689492e4159b7cbe04/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:29.241896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492e28e-2b18-4ca8-9e70-990eac855a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f01b673cff84a7995346e849e05662d, user_id=83327ce96ae348689492e4159b7cbe04, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:29.245012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492e28e-2b18-4ca8-9e70-990eac855a39 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:29.256545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1288/2524] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:11:29 2022] PUT /identity/v3/projects/8f01b673cff84a7995346e849e05662d/users/83327ce96ae348689492e4159b7cbe04/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:29.273663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34798c-8afb-4770-9018-c7c035957b64 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:29.274220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34798c-8afb-4770-9018-c7c035957b64 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:29.274220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34798c-8afb-4770-9018-c7c035957b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:29.310098 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-ed34798c-8afb-4770-9018-c7c035957b64 None None] MFA Rules not processed for user `83327ce96ae348689492e4159b7cbe04`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:29.311398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ed34798c-8afb-4770-9018-c7c035957b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:29.398512 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1237/2525] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:11:29.411750 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492e28e-2b18-4ca8-9e70-990eac855a39 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:29.413645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2492e28e-2b18-4ca8-9e70-990eac855a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:29.442210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:29.480670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492e28e-2b18-4ca8-9e70-990eac855a39 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:29.486323 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606d364-5bbd-4991-84ca-787c8540433c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:29.486817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606d364-5bbd-4991-84ca-787c8540433c service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:29.487073 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606d364-5bbd-4991-84ca-787c8540433c service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:29.487358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606d364-5bbd-4991-84ca-787c8540433c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:29.498148 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606d364-5bbd-4991-84ca-787c8540433c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:29.499755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606d364-5bbd-4991-84ca-787c8540433c service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:29.511428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1289/2526] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:11:30.820652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34798c-8afb-4770-9018-c7c035957b64 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:30.887172 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34798c-8afb-4770-9018-c7c035957b64 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:30.891784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:30.892440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:30.892716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:30.892984 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] PATH_INFO: `/v3/users/2f57112ad4754f09add6a84d588bf873` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:30.894192 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f57112ad4754f09add6a84d588bf873)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:30.896128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:31.157776 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f57112ad4754f09add6a84d588bf873'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:31.165638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f57112ad4754f09add6a84d588bf873'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:31.166394 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f57112ad4754f09add6a84d588bf873'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:31.291062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f57112ad4754f09add6a84d588bf873'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:31.307817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f57112ad4754f09add6a84d588bf873'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:31.315564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1238/2527] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:11:30 2022] DELETE /identity/v3/users/2f57112ad4754f09add6a84d588bf873 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:31.321979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc88fee-3708-4209-b771-9b327bcec072 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:31.323338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc88fee-3708-4209-b771-9b327bcec072 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:31.323506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc88fee-3708-4209-b771-9b327bcec072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:31.344948 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-3bc88fee-3708-4209-b771-9b327bcec072 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:31.346258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3bc88fee-3708-4209-b771-9b327bcec072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:31.414830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1290/2528] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:31.425370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:31.427198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:31.433868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:31.502048 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f4b5d-243a-4acb-890f-8d6397c3c5d6 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:31.509443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3ffa7-4f31-490c-bbc5-6a3bdf1983d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:31.511329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3ffa7-4f31-490c-bbc5-6a3bdf1983d7 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:31.511816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3ffa7-4f31-490c-bbc5-6a3bdf1983d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:31.512339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3ffa7-4f31-490c-bbc5-6a3bdf1983d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:31.529327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3ffa7-4f31-490c-bbc5-6a3bdf1983d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:31.532160 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3ffa7-4f31-490c-bbc5-6a3bdf1983d7 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:31.548555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1239/2529] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:11:32.109926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc88fee-3708-4209-b771-9b327bcec072 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:32.111508 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3bc88fee-3708-4209-b771-9b327bcec072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:32.125538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc88fee-3708-4209-b771-9b327bcec072 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:32.131253 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:32.132042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:32.132381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:32.132665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] PATH_INFO: `/v3/projects/e2d28572de504e558e7d96f854fcda9e` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:32.134028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d28572de504e558e7d96f854fcda9e)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:32.136961 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:32.287031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d28572de504e558e7d96f854fcda9e', 'request_id': 'req-429d1067-c1ff-43bf-b6a3-162e59512db5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:32.297186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d28572de504e558e7d96f854fcda9e', 'request_id': 'req-429d1067-c1ff-43bf-b6a3-162e59512db5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:32.382106 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d28572de504e558e7d96f854fcda9e', 'request_id': 'req-429d1067-c1ff-43bf-b6a3-162e59512db5'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:32.397321 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] The token cache is being invalidate because project e2d28572de504e558e7d96f854fcda9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:11:32.397781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:32.398574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:32.400259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1291/2530] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:11:32 2022] DELETE /identity/v3/projects/e2d28572de504e558e7d96f854fcda9e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:11:32.404900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3ffa7-4f31-490c-bbc5-6a3bdf1983d7 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:32.416871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3ffa7-4f31-490c-bbc5-6a3bdf1983d7 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:32.482693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:32.483373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:32.483654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:32.483937 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] PATH_INFO: `/v3/users/83327ce96ae348689492e4159b7cbe04` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:32.485145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=83327ce96ae348689492e4159b7cbe04)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:32.487190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:32.687845 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83327ce96ae348689492e4159b7cbe04'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:32.696544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83327ce96ae348689492e4159b7cbe04'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:32.697312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83327ce96ae348689492e4159b7cbe04'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:32.788103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83327ce96ae348689492e4159b7cbe04'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:32.806465 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-168348fd-5c63-4116-b1d9-f1ade9da9afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83327ce96ae348689492e4159b7cbe04'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:32.815091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1240/2531] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:11:32 2022] DELETE /identity/v3/users/83327ce96ae348689492e4159b7cbe04 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:33.431066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:33.433105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:33.453730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d1067-c1ff-43bf-b6a3-162e59512db5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:33.521078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:33.539842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:33.540589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:33.540874 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:33.541155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] PATH_INFO: `/v3/projects/8f01b673cff84a7995346e849e05662d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:33.542406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f01b673cff84a7995346e849e05662d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:33.544564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:33.668939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f01b673cff84a7995346e849e05662d', 'request_id': 'req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:33.677186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f01b673cff84a7995346e849e05662d', 'request_id': 'req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:33.793882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f01b673cff84a7995346e849e05662d', 'request_id': 'req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:33.806554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] The token cache is being invalidate because project 8f01b673cff84a7995346e849e05662d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:11:33.807168 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:33.807985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1f55ad-b5c7-4f85-94bd-43517dce49e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:33.812850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1292/2532] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:11:33 2022] DELETE /identity/v3/projects/8f01b673cff84a7995346e849e05662d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:11:33.815480 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd823592-a142-4ddf-8de6-1be0346cfc7a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:33.816016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd823592-a142-4ddf-8de6-1be0346cfc7a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:33.816118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd823592-a142-4ddf-8de6-1be0346cfc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:33.839235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-bd823592-a142-4ddf-8de6-1be0346cfc7a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:33.840682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-bd823592-a142-4ddf-8de6-1be0346cfc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:33.863239 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:33.910040 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1241/2533] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:33.918999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292ec17-decb-4ea2-91e4-a6933182f4fb None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:33.919545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292ec17-decb-4ea2-91e4-a6933182f4fb None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:33.923501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292ec17-decb-4ea2-91e4-a6933182f4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:33.940449 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1292ec17-decb-4ea2-91e4-a6933182f4fb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:33.941744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1292ec17-decb-4ea2-91e4-a6933182f4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:33.952009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1293/2534] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:33.956655 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd823592-a142-4ddf-8de6-1be0346cfc7a None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:33.967370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd823592-a142-4ddf-8de6-1be0346cfc7a None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:33.972406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56421cc2-8f01-47eb-956b-89224c588890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:33.973052 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56421cc2-8f01-47eb-956b-89224c588890 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:33.973330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56421cc2-8f01-47eb-956b-89224c588890 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:33.973620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56421cc2-8f01-47eb-956b-89224c588890 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:33.974386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56421cc2-8f01-47eb-956b-89224c588890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:11:33.974804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56421cc2-8f01-47eb-956b-89224c588890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:33.976346 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56421cc2-8f01-47eb-956b-89224c588890 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:33.990105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1242/2535] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:11:33.996857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292ec17-decb-4ea2-91e4-a6933182f4fb None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:34.010544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292ec17-decb-4ea2-91e4-a6933182f4fb None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:34.016122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1dbf12-6584-480d-bff0-ed2eec0f3728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:34.016813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1dbf12-6584-480d-bff0-ed2eec0f3728 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:34.017153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1dbf12-6584-480d-bff0-ed2eec0f3728 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:34.017786 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1dbf12-6584-480d-bff0-ed2eec0f3728 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:34.018825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1dbf12-6584-480d-bff0-ed2eec0f3728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:34.021344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1dbf12-6584-480d-bff0-ed2eec0f3728 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:34.167286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1294/2536] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:11:33 2022] POST /identity/v3/projects => generated 357 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:11:34.177379 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56421cc2-8f01-47eb-956b-89224c588890 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:34.187884 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56421cc2-8f01-47eb-956b-89224c588890 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:34.260586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc8999-5f41-418a-9d42-0a6181e2388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:34.261209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc8999-5f41-418a-9d42-0a6181e2388c admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:34.261503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc8999-5f41-418a-9d42-0a6181e2388c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:34.261798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc8999-5f41-418a-9d42-0a6181e2388c admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:34.262495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc8999-5f41-418a-9d42-0a6181e2388c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:34.264592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc8999-5f41-418a-9d42-0a6181e2388c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:34.394847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1243/2537] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:11:34 2022] POST /identity/v3/users => generated 279 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:11:34.399689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1dbf12-6584-480d-bff0-ed2eec0f3728 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:34.412297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1dbf12-6584-480d-bff0-ed2eec0f3728 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:34.417685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87d092-d2e2-4f28-af20-4c8832ff33fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:34.418328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87d092-d2e2-4f28-af20-4c8832ff33fd admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:34.418608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87d092-d2e2-4f28-af20-4c8832ff33fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:34.418876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87d092-d2e2-4f28-af20-4c8832ff33fd admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:34.419546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87d092-d2e2-4f28-af20-4c8832ff33fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:34.421386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87d092-d2e2-4f28-af20-4c8832ff33fd admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:34.442288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1295/2538] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:11:34.446798 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc8999-5f41-418a-9d42-0a6181e2388c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:34.459571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc8999-5f41-418a-9d42-0a6181e2388c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:34.469993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e654b2-e294-49a2-8289-205eaf81f41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:34.469993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e654b2-e294-49a2-8289-205eaf81f41b admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:34.469993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e654b2-e294-49a2-8289-205eaf81f41b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:34.469993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e654b2-e294-49a2-8289-205eaf81f41b admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:34.470648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e654b2-e294-49a2-8289-205eaf81f41b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:34.470648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e654b2-e294-49a2-8289-205eaf81f41b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:34.481725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1244/2539] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:11:34.486473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87d092-d2e2-4f28-af20-4c8832ff33fd admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:34.498072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87d092-d2e2-4f28-af20-4c8832ff33fd admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:34.503598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38975d-e6a1-4118-ada1-bb61db553bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:34.504423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38975d-e6a1-4118-ada1-bb61db553bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:34.504813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38975d-e6a1-4118-ada1-bb61db553bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:34.505151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38975d-e6a1-4118-ada1-bb61db553bbe admin admin] PATH_INFO: `/v3/projects/5f8f1a7ba2c4472c9a2b9797283fd4b6/users/639a2224073747cb9ff12d4aebeef5a4/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:34.524325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38975d-e6a1-4118-ada1-bb61db553bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8f1a7ba2c4472c9a2b9797283fd4b6, user_id=639a2224073747cb9ff12d4aebeef5a4, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:34.526898 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38975d-e6a1-4118-ada1-bb61db553bbe admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:34.637377 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1296/2540] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:11:34 2022] PUT /identity/v3/projects/5f8f1a7ba2c4472c9a2b9797283fd4b6/users/639a2224073747cb9ff12d4aebeef5a4/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:34.654642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7919b8-e1b5-4c52-8aed-94c2d1bcb9a8 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:34.655046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7919b8-e1b5-4c52-8aed-94c2d1bcb9a8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:34.655330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7919b8-e1b5-4c52-8aed-94c2d1bcb9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:34.697330 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-3d7919b8-e1b5-4c52-8aed-94c2d1bcb9a8 None None] MFA Rules not processed for user `639a2224073747cb9ff12d4aebeef5a4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:34.699151 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3d7919b8-e1b5-4c52-8aed-94c2d1bcb9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:34.790121 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1245/2541] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:34.794828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38975d-e6a1-4118-ada1-bb61db553bbe admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:34.887989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38975d-e6a1-4118-ada1-bb61db553bbe admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:34.894790 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dc14c-90ac-4d20-a2a4-6074ff1bfe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:34.895700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dc14c-90ac-4d20-a2a4-6074ff1bfe4a admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:34.896107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dc14c-90ac-4d20-a2a4-6074ff1bfe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:34.896599 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dc14c-90ac-4d20-a2a4-6074ff1bfe4a admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:34.897638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74dc14c-90ac-4d20-a2a4-6074ff1bfe4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:34.899992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74dc14c-90ac-4d20-a2a4-6074ff1bfe4a admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:35.038881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:35.046405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1297/2542] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:11:34 2022] POST /identity/v3/projects => generated 357 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:11:35.058969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7919b8-e1b5-4c52-8aed-94c2d1bcb9a8 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:35.070715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7919b8-e1b5-4c52-8aed-94c2d1bcb9a8 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:35.089704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:35.137904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649198d-dc58-4ec2-a220-bfc32e6e45db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:35.138528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649198d-dc58-4ec2-a220-bfc32e6e45db admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:35.138814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649198d-dc58-4ec2-a220-bfc32e6e45db admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:35.139083 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649198d-dc58-4ec2-a220-bfc32e6e45db admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:35.139748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649198d-dc58-4ec2-a220-bfc32e6e45db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:35.141911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649198d-dc58-4ec2-a220-bfc32e6e45db admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:35.263200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1246/2543] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:11:35 2022] POST /identity/v3/users => generated 285 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:11:35.267527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dc14c-90ac-4d20-a2a4-6074ff1bfe4a admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:35.278813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dc14c-90ac-4d20-a2a4-6074ff1bfe4a admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:35.283540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df890c-2a8a-4a85-89b9-a2ad5f08639f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:35.284160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df890c-2a8a-4a85-89b9-a2ad5f08639f admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:35.284473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df890c-2a8a-4a85-89b9-a2ad5f08639f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:35.284809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df890c-2a8a-4a85-89b9-a2ad5f08639f admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:35.285461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df890c-2a8a-4a85-89b9-a2ad5f08639f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:35.287083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df890c-2a8a-4a85-89b9-a2ad5f08639f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:35.313009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1298/2544] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:11:35.314384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649198d-dc58-4ec2-a220-bfc32e6e45db admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:35.326356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649198d-dc58-4ec2-a220-bfc32e6e45db admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:35.331505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd6ba1-c6de-49d8-92c8-0326c9eafed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:35.332218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd6ba1-c6de-49d8-92c8-0326c9eafed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:35.332507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd6ba1-c6de-49d8-92c8-0326c9eafed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:35.332790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd6ba1-c6de-49d8-92c8-0326c9eafed1 admin admin] PATH_INFO: `/v3/projects/0b07134a2f0240138bf2de7eaddc1401/users/2b65e4c079844096995827e87ba2382e/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:35.353451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd6ba1-c6de-49d8-92c8-0326c9eafed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b07134a2f0240138bf2de7eaddc1401, user_id=2b65e4c079844096995827e87ba2382e, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:35.357424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd6ba1-c6de-49d8-92c8-0326c9eafed1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:35.483308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1247/2545] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:11:35 2022] PUT /identity/v3/projects/0b07134a2f0240138bf2de7eaddc1401/users/2b65e4c079844096995827e87ba2382e/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:35.487664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df890c-2a8a-4a85-89b9-a2ad5f08639f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:35.499568 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df890c-2a8a-4a85-89b9-a2ad5f08639f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:35.572046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac83bfd-c1a8-4b0b-aed4-aa76818fdcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:35.573626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac83bfd-c1a8-4b0b-aed4-aa76818fdcfe admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:35.574127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac83bfd-c1a8-4b0b-aed4-aa76818fdcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:35.574607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac83bfd-c1a8-4b0b-aed4-aa76818fdcfe admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:35.575789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac83bfd-c1a8-4b0b-aed4-aa76818fdcfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:35.578926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac83bfd-c1a8-4b0b-aed4-aa76818fdcfe admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:35.601820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1299/2546] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:11:35.604738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd6ba1-c6de-49d8-92c8-0326c9eafed1 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:35.616235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd6ba1-c6de-49d8-92c8-0326c9eafed1 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:35.621672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf9883-1b7a-454d-b414-488d438a25ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:35.622358 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf9883-1b7a-454d-b414-488d438a25ed admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:35.622641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf9883-1b7a-454d-b414-488d438a25ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:35.622923 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf9883-1b7a-454d-b414-488d438a25ed admin admin] PATH_INFO: `/v3/projects/0b07134a2f0240138bf2de7eaddc1401/users/2b65e4c079844096995827e87ba2382e/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:35.625056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf9883-1b7a-454d-b414-488d438a25ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b07134a2f0240138bf2de7eaddc1401, user_id=2b65e4c079844096995827e87ba2382e, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:35.630201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf9883-1b7a-454d-b414-488d438a25ed admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:35.641290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1248/2547] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:11:35 2022] PUT /identity/v3/projects/0b07134a2f0240138bf2de7eaddc1401/users/2b65e4c079844096995827e87ba2382e/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:35.653967 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259755f-015a-4142-aec8-262fae5007b8 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:35.658989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259755f-015a-4142-aec8-262fae5007b8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:35.658989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259755f-015a-4142-aec8-262fae5007b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:35.688462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9259755f-015a-4142-aec8-262fae5007b8 None None] MFA Rules not processed for user `2b65e4c079844096995827e87ba2382e`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:35.689775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9259755f-015a-4142-aec8-262fae5007b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:35.776842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1300/2548] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:35.790405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf9883-1b7a-454d-b414-488d438a25ed admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:35.794182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-fccf9883-1b7a-454d-b414-488d438a25ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:35.886103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf9883-1b7a-454d-b414-488d438a25ed admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:35.890175 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710a4a2-c34f-4634-b29c-c4b5dc7ac1e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:35.890846 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710a4a2-c34f-4634-b29c-c4b5dc7ac1e5 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:35.891184 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710a4a2-c34f-4634-b29c-c4b5dc7ac1e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:35.891510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710a4a2-c34f-4634-b29c-c4b5dc7ac1e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:35.902576 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710a4a2-c34f-4634-b29c-c4b5dc7ac1e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:35.907996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710a4a2-c34f-4634-b29c-c4b5dc7ac1e5 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:35.915740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1249/2549] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:11:36.154910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259755f-015a-4142-aec8-262fae5007b8 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:36.168863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259755f-015a-4142-aec8-262fae5007b8 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:36.173460 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183c7db-2384-4bb0-8102-e8817866c190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:36.174183 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183c7db-2384-4bb0-8102-e8817866c190 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:36.174674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183c7db-2384-4bb0-8102-e8817866c190 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:36.175078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183c7db-2384-4bb0-8102-e8817866c190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:36.240601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183c7db-2384-4bb0-8102-e8817866c190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:36.242366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183c7db-2384-4bb0-8102-e8817866c190 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:36.253539 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1301/2550] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:11:45.910259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:46.248197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:11:58.453962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710a4a2-c34f-4634-b29c-c4b5dc7ac1e5 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:58.456108 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-9710a4a2-c34f-4634-b29c-c4b5dc7ac1e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:58.580616 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710a4a2-c34f-4634-b29c-c4b5dc7ac1e5 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:58.589095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:58.590199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:58.590695 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:58.591196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] PATH_INFO: `/v3/users/c91a2dbf9e24494883186f0e2f6ff4d2` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:58.593371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91a2dbf9e24494883186f0e2f6ff4d2)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:58.597153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:58.830095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91a2dbf9e24494883186f0e2f6ff4d2'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:58.838742 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91a2dbf9e24494883186f0e2f6ff4d2'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:58.839492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91a2dbf9e24494883186f0e2f6ff4d2'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:58.942896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91a2dbf9e24494883186f0e2f6ff4d2'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:58.959244 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91a2dbf9e24494883186f0e2f6ff4d2'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:58.966347 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1250/2551] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:11:58 2022] DELETE /identity/v3/users/c91a2dbf9e24494883186f0e2f6ff4d2 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:11:58.973059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2efc3d-78a6-4a3d-8362-32e950594197 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:58.973650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2efc3d-78a6-4a3d-8362-32e950594197 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:58.973759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2efc3d-78a6-4a3d-8362-32e950594197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:58.996753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-ab2efc3d-78a6-4a3d-8362-32e950594197 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:11:58.998036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ab2efc3d-78a6-4a3d-8362-32e950594197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:59.072192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1302/2552] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:11:59.082540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:59.084388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:59.166930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d13e7-dd6e-4c32-9bd0-82fa99a60b6c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:59.170732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638229a8-86d7-437e-875d-e1154e456616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:59.171348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638229a8-86d7-437e-875d-e1154e456616 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:59.171642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638229a8-86d7-437e-875d-e1154e456616 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:59.172001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638229a8-86d7-437e-875d-e1154e456616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:59.182841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638229a8-86d7-437e-875d-e1154e456616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:59.184433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638229a8-86d7-437e-875d-e1154e456616 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:59.195274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1251/2553] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:11:59.733581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2efc3d-78a6-4a3d-8362-32e950594197 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:11:59.737462 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ab2efc3d-78a6-4a3d-8362-32e950594197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:11:59.754160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2efc3d-78a6-4a3d-8362-32e950594197 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:11:59.759210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:11:59.759883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:11:59.760162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:11:59.760473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] PATH_INFO: `/v3/projects/ef21a298b8374520aad4854e6dd343db` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:11:59.762066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef21a298b8374520aad4854e6dd343db)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:11:59.764674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:11:59.906822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef21a298b8374520aad4854e6dd343db', 'request_id': 'req-76a705ba-198b-4b92-af53-197c3066e5d6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:59.917382 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef21a298b8374520aad4854e6dd343db', 'request_id': 'req-76a705ba-198b-4b92-af53-197c3066e5d6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:11:59.998807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef21a298b8374520aad4854e6dd343db', 'request_id': 'req-76a705ba-198b-4b92-af53-197c3066e5d6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:00.013137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] The token cache is being invalidate because project ef21a298b8374520aad4854e6dd343db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:00.013548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:00.018284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:00.018284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1303/2554] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:11:59 2022] DELETE /identity/v3/projects/ef21a298b8374520aad4854e6dd343db => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:00.021796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638229a8-86d7-437e-875d-e1154e456616 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:00.032869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638229a8-86d7-437e-875d-e1154e456616 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:00.056522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:00.101005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:00.101653 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:00.101940 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:00.102218 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] PATH_INFO: `/v3/users/b86dc78b4691490393ae852df86f070b` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:00.104022 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86dc78b4691490393ae852df86f070b)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:00.106157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:00.340468 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86dc78b4691490393ae852df86f070b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:00.348690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86dc78b4691490393ae852df86f070b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:00.349467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86dc78b4691490393ae852df86f070b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:00.534217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86dc78b4691490393ae852df86f070b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:00.560369 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-55af4b66-6d73-48b4-ae39-47d2645f2876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86dc78b4691490393ae852df86f070b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:00.572739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1252/2555] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:00 2022] DELETE /identity/v3/users/b86dc78b4691490393ae852df86f070b => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:01.189537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:01.191979 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:01.213603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a705ba-198b-4b92-af53-197c3066e5d6 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:01.301610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:01.302699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:01.303210 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:01.303708 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] PATH_INFO: `/v3/projects/8eb1d0e07fc64aef8209fd24c5d27d6f` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:01.305939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb1d0e07fc64aef8209fd24c5d27d6f)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:01.310486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:01.464060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:01.482170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb1d0e07fc64aef8209fd24c5d27d6f', 'request_id': 'req-b23834bc-85a3-45fb-8591-50289b0c5f4f'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:01.487588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb1d0e07fc64aef8209fd24c5d27d6f', 'request_id': 'req-b23834bc-85a3-45fb-8591-50289b0c5f4f'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:01.569899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb1d0e07fc64aef8209fd24c5d27d6f', 'request_id': 'req-b23834bc-85a3-45fb-8591-50289b0c5f4f'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:01.583509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] The token cache is being invalidate because project 8eb1d0e07fc64aef8209fd24c5d27d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:01.583804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:01.584481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-b23834bc-85a3-45fb-8591-50289b0c5f4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:01.586061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1304/2556] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:01 2022] DELETE /identity/v3/projects/8eb1d0e07fc64aef8209fd24c5d27d6f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:01.591156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc77253-807f-4562-b5ac-b6ea4b969282 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:01.591851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc77253-807f-4562-b5ac-b6ea4b969282 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:01.591851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc77253-807f-4562-b5ac-b6ea4b969282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:01.614817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-9cc77253-807f-4562-b5ac-b6ea4b969282 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:01.616340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-9cc77253-807f-4562-b5ac-b6ea4b969282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:01.686375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1253/2557] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:01.695811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e158a3b-4433-4628-8f8f-e7034075d718 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:01.696230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e158a3b-4433-4628-8f8f-e7034075d718 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:01.697177 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e158a3b-4433-4628-8f8f-e7034075d718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:01.719393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4e158a3b-4433-4628-8f8f-e7034075d718 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:01.720543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4e158a3b-4433-4628-8f8f-e7034075d718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:01.728951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1305/2558] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:01.733390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc77253-807f-4562-b5ac-b6ea4b969282 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:01.744304 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc77253-807f-4562-b5ac-b6ea4b969282 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:01.749331 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f444d5-7bc1-499b-8a22-7f37ab8f8c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:01.749947 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f444d5-7bc1-499b-8a22-7f37ab8f8c9e admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:01.750225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f444d5-7bc1-499b-8a22-7f37ab8f8c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:01.750502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f444d5-7bc1-499b-8a22-7f37ab8f8c9e admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:01.751167 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f444d5-7bc1-499b-8a22-7f37ab8f8c9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:12:01.751724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f444d5-7bc1-499b-8a22-7f37ab8f8c9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:01.753417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f444d5-7bc1-499b-8a22-7f37ab8f8c9e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:01.772971 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1254/2559] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:12:01.775384 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e158a3b-4433-4628-8f8f-e7034075d718 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:01.792023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e158a3b-4433-4628-8f8f-e7034075d718 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:01.805431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d429ff-8723-48c4-8329-4c0c9cfa62ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:01.807191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d429ff-8723-48c4-8329-4c0c9cfa62ea admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:01.807824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d429ff-8723-48c4-8329-4c0c9cfa62ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:01.808818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d429ff-8723-48c4-8329-4c0c9cfa62ea admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:01.809839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d429ff-8723-48c4-8329-4c0c9cfa62ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:01.813235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d429ff-8723-48c4-8329-4c0c9cfa62ea admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:01.915673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1306/2560] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:01 2022] POST /identity/v3/projects => generated 393 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:01.920050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f444d5-7bc1-499b-8a22-7f37ab8f8c9e admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:01.931519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f444d5-7bc1-499b-8a22-7f37ab8f8c9e admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:01.967506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:02.012669 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210b95c-549b-4c4e-96b8-cf6b8cf66c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:02.013512 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210b95c-549b-4c4e-96b8-cf6b8cf66c38 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:02.013985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210b95c-549b-4c4e-96b8-cf6b8cf66c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:02.014456 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210b95c-549b-4c4e-96b8-cf6b8cf66c38 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:02.015460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210b95c-549b-4c4e-96b8-cf6b8cf66c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:02.018110 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210b95c-549b-4c4e-96b8-cf6b8cf66c38 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:02.128873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1255/2561] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:01 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:02.133438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d429ff-8723-48c4-8329-4c0c9cfa62ea admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:02.144517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d429ff-8723-48c4-8329-4c0c9cfa62ea admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:02.149603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd30110-c9d0-4517-82b7-e892bcb271ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:02.150236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd30110-c9d0-4517-82b7-e892bcb271ba admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:02.150539 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd30110-c9d0-4517-82b7-e892bcb271ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:02.150778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd30110-c9d0-4517-82b7-e892bcb271ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:02.151430 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd30110-c9d0-4517-82b7-e892bcb271ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:02.153086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd30110-c9d0-4517-82b7-e892bcb271ba admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:02.164395 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1307/2562] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:02.169586 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210b95c-549b-4c4e-96b8-cf6b8cf66c38 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:02.181301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210b95c-549b-4c4e-96b8-cf6b8cf66c38 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:02.186114 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bfa79-975c-4a65-9fde-65076b62c681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:02.186758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bfa79-975c-4a65-9fde-65076b62c681 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:02.187038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bfa79-975c-4a65-9fde-65076b62c681 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:02.187376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bfa79-975c-4a65-9fde-65076b62c681 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:02.188032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bfa79-975c-4a65-9fde-65076b62c681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:02.189704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bfa79-975c-4a65-9fde-65076b62c681 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:02.201804 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1256/2563] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:02.214513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd30110-c9d0-4517-82b7-e892bcb271ba admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:02.227050 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd30110-c9d0-4517-82b7-e892bcb271ba admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:02.233097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829079df-0f70-412a-a838-6874d00c0804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:02.233824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829079df-0f70-412a-a838-6874d00c0804 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:02.234157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829079df-0f70-412a-a838-6874d00c0804 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:02.234499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829079df-0f70-412a-a838-6874d00c0804 admin admin] PATH_INFO: `/v3/projects/7ade57641e144d489592c1b14503000b/users/dbc89e2bc58949c299c67d671b212fab/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:02.256759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829079df-0f70-412a-a838-6874d00c0804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ade57641e144d489592c1b14503000b, user_id=dbc89e2bc58949c299c67d671b212fab, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:02.258984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829079df-0f70-412a-a838-6874d00c0804 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:02.281903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bfa79-975c-4a65-9fde-65076b62c681 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:02.283707 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c58bfa79-975c-4a65-9fde-65076b62c681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:02.299134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bfa79-975c-4a65-9fde-65076b62c681 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:02.304422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:02.305067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:02.305340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:02.305654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] PATH_INFO: `/v3/users/639a2224073747cb9ff12d4aebeef5a4` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:02.306782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] RBAC: Authorizing `identity:delete_user(user_id=639a2224073747cb9ff12d4aebeef5a4)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:02.308757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:02.404721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1308/2564] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:02 2022] PUT /identity/v3/projects/7ade57641e144d489592c1b14503000b/users/dbc89e2bc58949c299c67d671b212fab/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:02.416628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430469a-9947-4e60-8ece-36ec210cb0de None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:02.416933 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430469a-9947-4e60-8ece-36ec210cb0de None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:02.417165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430469a-9947-4e60-8ece-36ec210cb0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:02.448712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b430469a-9947-4e60-8ece-36ec210cb0de None None] MFA Rules not processed for user `dbc89e2bc58949c299c67d671b212fab`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:02.449990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b430469a-9947-4e60-8ece-36ec210cb0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:02.532478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1309/2565] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:02.537239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430469a-9947-4e60-8ece-36ec210cb0de None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:02.575452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '639a2224073747cb9ff12d4aebeef5a4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:02.591942 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '639a2224073747cb9ff12d4aebeef5a4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:02.592705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:02.593234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '639a2224073747cb9ff12d4aebeef5a4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:02.624101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430469a-9947-4e60-8ece-36ec210cb0de None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:02.629644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b87631-9415-414e-b2f3-955e0b8315c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:02.630294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b87631-9415-414e-b2f3-955e0b8315c5 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:02.630562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b87631-9415-414e-b2f3-955e0b8315c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:02.630841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b87631-9415-414e-b2f3-955e0b8315c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:02.631649 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b87631-9415-414e-b2f3-955e0b8315c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:02.634236 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b87631-9415-414e-b2f3-955e0b8315c5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:02.710595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '639a2224073747cb9ff12d4aebeef5a4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:02.727604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-54fddafb-f562-4cbb-94b8-ab0d256e339a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '639a2224073747cb9ff12d4aebeef5a4'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:02.734972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1257/2566] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:02 2022] DELETE /identity/v3/users/639a2224073747cb9ff12d4aebeef5a4 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:02.740782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9479f69-4f6e-4705-8ac8-86954306ba3f None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:02.741600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9479f69-4f6e-4705-8ac8-86954306ba3f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:02.742067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9479f69-4f6e-4705-8ac8-86954306ba3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:02.757331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1310/2567] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:02 2022] POST /identity/v3/projects => generated 391 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:02.759295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b87631-9415-414e-b2f3-955e0b8315c5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:02.761558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-04b87631-9415-414e-b2f3-955e0b8315c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:02.771032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b9479f69-4f6e-4705-8ac8-86954306ba3f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:02.772360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b9479f69-4f6e-4705-8ac8-86954306ba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:02.867056 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b87631-9415-414e-b2f3-955e0b8315c5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:02.874709 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7a17d-c199-47c1-825d-bea3f67766f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:02.875721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7a17d-c199-47c1-825d-bea3f67766f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:02.876191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7a17d-c199-47c1-825d-bea3f67766f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:02.876930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7a17d-c199-47c1-825d-bea3f67766f9 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:02.878045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7a17d-c199-47c1-825d-bea3f67766f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:02.878257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1258/2568] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:02.879444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9479f69-4f6e-4705-8ac8-86954306ba3f None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:02.882589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b9479f69-4f6e-4705-8ac8-86954306ba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:02.882705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7a17d-c199-47c1-825d-bea3f67766f9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:02.910699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:02.954294 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9479f69-4f6e-4705-8ac8-86954306ba3f None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:02.958251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a123b95-f2e4-4cc7-9014-bb07462c1e9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:02.958904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a123b95-f2e4-4cc7-9014-bb07462c1e9a service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:02.959171 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a123b95-f2e4-4cc7-9014-bb07462c1e9a service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:02.959450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a123b95-f2e4-4cc7-9014-bb07462c1e9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:02.961332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4a123b95-f2e4-4cc7-9014-bb07462c1e9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:03.008601 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1311/2569] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:02 2022] POST /identity/v3/users => generated 302 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:03.010817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7a17d-c199-47c1-825d-bea3f67766f9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:03.012623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-36d7a17d-c199-47c1-825d-bea3f67766f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:03.061886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a123b95-f2e4-4cc7-9014-bb07462c1e9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:03.061886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a123b95-f2e4-4cc7-9014-bb07462c1e9a service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:03.069462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1259/2570] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:03.070370 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a123b95-f2e4-4cc7-9014-bb07462c1e9a service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:03.080880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7a17d-c199-47c1-825d-bea3f67766f9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:03.085311 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec0463-db0e-4aab-b29d-e77ef1f512e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:03.086095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0463-db0e-4aab-b29d-e77ef1f512e2 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:03.086452 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0463-db0e-4aab-b29d-e77ef1f512e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:03.086828 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0463-db0e-4aab-b29d-e77ef1f512e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:03.087908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a123b95-f2e4-4cc7-9014-bb07462c1e9a service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:03.095404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef37e4c-afd3-4a34-97cd-87f90d5c42a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:03.096118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef37e4c-afd3-4a34-97cd-87f90d5c42a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:03.096497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef37e4c-afd3-4a34-97cd-87f90d5c42a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:03.096869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef37e4c-afd3-4a34-97cd-87f90d5c42a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:03.097631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef37e4c-afd3-4a34-97cd-87f90d5c42a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:03.099426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef37e4c-afd3-4a34-97cd-87f90d5c42a4 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:03.103109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec0463-db0e-4aab-b29d-e77ef1f512e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:03.110383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec0463-db0e-4aab-b29d-e77ef1f512e2 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:03.115793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1260/2571] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:03.116852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef37e4c-afd3-4a34-97cd-87f90d5c42a4 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:03.123840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1312/2572] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:03.128817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef37e4c-afd3-4a34-97cd-87f90d5c42a4 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:03.135318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ad719-cc1a-4a79-8b66-4d1e051801fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:03.136249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ad719-cc1a-4a79-8b66-4d1e051801fd admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:03.136523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ad719-cc1a-4a79-8b66-4d1e051801fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:03.136805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ad719-cc1a-4a79-8b66-4d1e051801fd admin admin] PATH_INFO: `/v3/projects/ed34e4d506cc4cb99167bb89a4be5414/users/53dae47352784ce78222d148a83d05a8/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:03.162349 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ad719-cc1a-4a79-8b66-4d1e051801fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed34e4d506cc4cb99167bb89a4be5414, user_id=53dae47352784ce78222d148a83d05a8, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:03.167994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ad719-cc1a-4a79-8b66-4d1e051801fd admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:03.340030 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1261/2573] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:03 2022] PUT /identity/v3/projects/ed34e4d506cc4cb99167bb89a4be5414/users/53dae47352784ce78222d148a83d05a8/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:03.345165 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec0463-db0e-4aab-b29d-e77ef1f512e2 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:03.360081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec0463-db0e-4aab-b29d-e77ef1f512e2 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:03.425327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:03.432331 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b8273-2ce4-4661-98e8-5597ba191f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:03.432988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b8273-2ce4-4661-98e8-5597ba191f10 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:03.433257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b8273-2ce4-4661-98e8-5597ba191f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:03.433526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b8273-2ce4-4661-98e8-5597ba191f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:03.434191 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b8273-2ce4-4661-98e8-5597ba191f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:03.435935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b8273-2ce4-4661-98e8-5597ba191f10 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:03.451049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1313/2574] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:03.456317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ad719-cc1a-4a79-8b66-4d1e051801fd admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:03.478839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ad719-cc1a-4a79-8b66-4d1e051801fd admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:03.488262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd74fd-35f3-44aa-b8d8-3b41dcc598ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:03.489271 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd74fd-35f3-44aa-b8d8-3b41dcc598ad admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:03.489766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd74fd-35f3-44aa-b8d8-3b41dcc598ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:03.490339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd74fd-35f3-44aa-b8d8-3b41dcc598ad admin admin] PATH_INFO: `/v3/projects/ed34e4d506cc4cb99167bb89a4be5414/users/53dae47352784ce78222d148a83d05a8/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:03.495165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd74fd-35f3-44aa-b8d8-3b41dcc598ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed34e4d506cc4cb99167bb89a4be5414, user_id=53dae47352784ce78222d148a83d05a8, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:03.501220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd74fd-35f3-44aa-b8d8-3b41dcc598ad admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:03.525714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1262/2575] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:03 2022] PUT /identity/v3/projects/ed34e4d506cc4cb99167bb89a4be5414/users/53dae47352784ce78222d148a83d05a8/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:03.537071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320deb2-b812-4b71-a047-6c92357472f8 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:03.537649 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320deb2-b812-4b71-a047-6c92357472f8 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:03.538143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320deb2-b812-4b71-a047-6c92357472f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:03.594575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-8320deb2-b812-4b71-a047-6c92357472f8 None None] MFA Rules not processed for user `53dae47352784ce78222d148a83d05a8`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:03.596440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8320deb2-b812-4b71-a047-6c92357472f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:03.706478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1314/2576] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:03 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:03.720046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd74fd-35f3-44aa-b8d8-3b41dcc598ad admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:03.723005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d5cd74fd-35f3-44aa-b8d8-3b41dcc598ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:03.791704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320deb2-b812-4b71-a047-6c92357472f8 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:03.795016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8320deb2-b812-4b71-a047-6c92357472f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:03.813355 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd74fd-35f3-44aa-b8d8-3b41dcc598ad admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:03.817516 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb742b-d6c0-4ab4-ac93-c13e20c72be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:03.818153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb742b-d6c0-4ab4-ac93-c13e20c72be4 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:03.818484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb742b-d6c0-4ab4-ac93-c13e20c72be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:03.818753 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb742b-d6c0-4ab4-ac93-c13e20c72be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:03.830614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb742b-d6c0-4ab4-ac93-c13e20c72be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:03.832416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb742b-d6c0-4ab4-ac93-c13e20c72be4 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:03.844412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1263/2577] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:03.910156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb742b-d6c0-4ab4-ac93-c13e20c72be4 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:03.915766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320deb2-b812-4b71-a047-6c92357472f8 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:03.920439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:03.921289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:03.921633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:03.922129 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] PATH_INFO: `/v3/projects/5f8f1a7ba2c4472c9a2b9797283fd4b6` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:03.924030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f8f1a7ba2c4472c9a2b9797283fd4b6)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:03.925526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb742b-d6c0-4ab4-ac93-c13e20c72be4 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:03.926335 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:03.932606 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea112c9-d37d-4ad8-9a4d-dfd3e3dfa532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:03.933640 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea112c9-d37d-4ad8-9a4d-dfd3e3dfa532 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:03.934080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea112c9-d37d-4ad8-9a4d-dfd3e3dfa532 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:03.934483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea112c9-d37d-4ad8-9a4d-dfd3e3dfa532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:03.937233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4ea112c9-d37d-4ad8-9a4d-dfd3e3dfa532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:03.996421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:04.059934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea112c9-d37d-4ad8-9a4d-dfd3e3dfa532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:04.062730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea112c9-d37d-4ad8-9a4d-dfd3e3dfa532 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:04.080682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1264/2578] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:04.137153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f8f1a7ba2c4472c9a2b9797283fd4b6', 'request_id': 'req-fe8c9215-728b-4ccd-ba5a-85f4f4663448'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:04.144543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f8f1a7ba2c4472c9a2b9797283fd4b6', 'request_id': 'req-fe8c9215-728b-4ccd-ba5a-85f4f4663448'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:04.244336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f8f1a7ba2c4472c9a2b9797283fd4b6', 'request_id': 'req-fe8c9215-728b-4ccd-ba5a-85f4f4663448'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:04.258008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] The token cache is being invalidate because project 5f8f1a7ba2c4472c9a2b9797283fd4b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:04.258295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:04.259533 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:04.261124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1315/2579] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:03 2022] DELETE /identity/v3/projects/5f8f1a7ba2c4472c9a2b9797283fd4b6 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:04.265310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea112c9-d37d-4ad8-9a4d-dfd3e3dfa532 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:04.277521 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea112c9-d37d-4ad8-9a4d-dfd3e3dfa532 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:04.348718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:04.349371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:04.349656 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:04.349940 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] PATH_INFO: `/v3/users/2b65e4c079844096995827e87ba2382e` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:04.351249 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b65e4c079844096995827e87ba2382e)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:04.353332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:04.574564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b65e4c079844096995827e87ba2382e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:04.583641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b65e4c079844096995827e87ba2382e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:04.584450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b65e4c079844096995827e87ba2382e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:04.827251 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b65e4c079844096995827e87ba2382e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:04.840726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-09c3297e-06bc-4f29-8a57-ef6c7c9b2117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b65e4c079844096995827e87ba2382e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:04.847414 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1265/2580] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:04 2022] DELETE /identity/v3/users/2b65e4c079844096995827e87ba2382e => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:05.453488 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:05.455523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:05.476681 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c9215-728b-4ccd-ba5a-85f4f4663448 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:05.487093 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:05.553810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:05.554525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:05.554808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:05.555094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] PATH_INFO: `/v3/projects/0b07134a2f0240138bf2de7eaddc1401` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:05.556390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b07134a2f0240138bf2de7eaddc1401)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:05.558367 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:05.722513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b07134a2f0240138bf2de7eaddc1401', 'request_id': 'req-4e1bc3c5-b31c-4a49-8415-008784c15bee'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:05.729334 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b07134a2f0240138bf2de7eaddc1401', 'request_id': 'req-4e1bc3c5-b31c-4a49-8415-008784c15bee'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:05.850342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b07134a2f0240138bf2de7eaddc1401', 'request_id': 'req-4e1bc3c5-b31c-4a49-8415-008784c15bee'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:05.871116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] The token cache is being invalidate because project 0b07134a2f0240138bf2de7eaddc1401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:05.871576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:05.872517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e1bc3c5-b31c-4a49-8415-008784c15bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:05.874197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1316/2581] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:05 2022] DELETE /identity/v3/projects/0b07134a2f0240138bf2de7eaddc1401 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:05.880476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8ddc5-6c80-44db-9ea9-73b78de7b20a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:05.881253 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8ddc5-6c80-44db-9ea9-73b78de7b20a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:05.881838 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8ddc5-6c80-44db-9ea9-73b78de7b20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:05.906731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-cac8ddc5-6c80-44db-9ea9-73b78de7b20a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:05.907992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-cac8ddc5-6c80-44db-9ea9-73b78de7b20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:05.969125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:05.994059 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1266/2582] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:06.007198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73252577-c8dd-4a19-8763-1dd3a39347c0 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:06.007555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73252577-c8dd-4a19-8763-1dd3a39347c0 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:06.007939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73252577-c8dd-4a19-8763-1dd3a39347c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:06.032521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-73252577-c8dd-4a19-8763-1dd3a39347c0 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:06.034121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-73252577-c8dd-4a19-8763-1dd3a39347c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:06.048107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1317/2583] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:06.064916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8ddc5-6c80-44db-9ea9-73b78de7b20a None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:06.078510 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8ddc5-6c80-44db-9ea9-73b78de7b20a None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:06.095018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abd1db-5baa-4a53-999d-be35afacb3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:06.099665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abd1db-5baa-4a53-999d-be35afacb3ee admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:06.099665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abd1db-5baa-4a53-999d-be35afacb3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:06.099665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abd1db-5baa-4a53-999d-be35afacb3ee admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:06.099665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abd1db-5baa-4a53-999d-be35afacb3ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:12:06.099665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abd1db-5baa-4a53-999d-be35afacb3ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:06.100550 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abd1db-5baa-4a53-999d-be35afacb3ee admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:06.123163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1267/2584] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:12:06.128976 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73252577-c8dd-4a19-8763-1dd3a39347c0 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:06.139028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73252577-c8dd-4a19-8763-1dd3a39347c0 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:06.144142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57296f61-e3b1-4cac-8d5a-9534251f1eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:06.144788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57296f61-e3b1-4cac-8d5a-9534251f1eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:06.145070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57296f61-e3b1-4cac-8d5a-9534251f1eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:06.145343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57296f61-e3b1-4cac-8d5a-9534251f1eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:06.146072 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57296f61-e3b1-4cac-8d5a-9534251f1eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:06.148130 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57296f61-e3b1-4cac-8d5a-9534251f1eb7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:06.348519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1318/2585] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:06 2022] POST /identity/v3/projects => generated 401 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:06.350162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abd1db-5baa-4a53-999d-be35afacb3ee admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:06.369648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abd1db-5baa-4a53-999d-be35afacb3ee admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:06.441139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff4375-82fa-4e75-831f-78fae87ff1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:06.441667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff4375-82fa-4e75-831f-78fae87ff1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:06.441950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff4375-82fa-4e75-831f-78fae87ff1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:06.442231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff4375-82fa-4e75-831f-78fae87ff1b0 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:06.443018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff4375-82fa-4e75-831f-78fae87ff1b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:06.445041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff4375-82fa-4e75-831f-78fae87ff1b0 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:06.614075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1268/2586] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:06 2022] POST /identity/v3/users => generated 301 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:06.615408 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57296f61-e3b1-4cac-8d5a-9534251f1eb7 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:06.635455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57296f61-e3b1-4cac-8d5a-9534251f1eb7 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:06.643653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0df96a-c65c-41bd-8f56-76a4dcdffd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:06.644742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0df96a-c65c-41bd-8f56-76a4dcdffd44 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:06.645229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0df96a-c65c-41bd-8f56-76a4dcdffd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:06.645692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0df96a-c65c-41bd-8f56-76a4dcdffd44 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:06.646824 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0df96a-c65c-41bd-8f56-76a4dcdffd44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:06.649924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0df96a-c65c-41bd-8f56-76a4dcdffd44 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:06.667178 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1319/2587] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:06.671877 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff4375-82fa-4e75-831f-78fae87ff1b0 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:06.683875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff4375-82fa-4e75-831f-78fae87ff1b0 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:06.689027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b4103-067f-41d7-9f72-781bd225829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:06.689657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4103-067f-41d7-9f72-781bd225829f admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:06.689961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4103-067f-41d7-9f72-781bd225829f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:06.690214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b4103-067f-41d7-9f72-781bd225829f admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:06.690963 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b4103-067f-41d7-9f72-781bd225829f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:06.692793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b4103-067f-41d7-9f72-781bd225829f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:06.708413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1269/2588] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:06.710407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0df96a-c65c-41bd-8f56-76a4dcdffd44 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:06.722789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0df96a-c65c-41bd-8f56-76a4dcdffd44 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:06.731926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7939130-9a95-465e-9db4-7d78338f00df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:06.733141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7939130-9a95-465e-9db4-7d78338f00df admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:06.733694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7939130-9a95-465e-9db4-7d78338f00df admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:06.734393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7939130-9a95-465e-9db4-7d78338f00df admin admin] PATH_INFO: `/v3/projects/0b9e841054e045c29a3fef523a8e79e0/users/9031ef5177d24512b18ad15c36cd6878/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:06.770277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7939130-9a95-465e-9db4-7d78338f00df admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9e841054e045c29a3fef523a8e79e0, user_id=9031ef5177d24512b18ad15c36cd6878, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:06.776681 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7939130-9a95-465e-9db4-7d78338f00df admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:06.875193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1320/2589] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:06 2022] PUT /identity/v3/projects/0b9e841054e045c29a3fef523a8e79e0/users/9031ef5177d24512b18ad15c36cd6878/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:06.893322 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91464d2c-837b-44aa-92b9-dc6490bb76bc None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:06.894004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91464d2c-837b-44aa-92b9-dc6490bb76bc None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:06.894004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91464d2c-837b-44aa-92b9-dc6490bb76bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:06.927485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-91464d2c-837b-44aa-92b9-dc6490bb76bc None None] MFA Rules not processed for user `9031ef5177d24512b18ad15c36cd6878`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:06.928836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-91464d2c-837b-44aa-92b9-dc6490bb76bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:07.010092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1270/2590] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:07.015520 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7939130-9a95-465e-9db4-7d78338f00df admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:07.097515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:07.135860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7939130-9a95-465e-9db4-7d78338f00df admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:07.144328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658b39a-e415-46c5-9225-a8519186689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:07.145379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658b39a-e415-46c5-9225-a8519186689d admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:07.146146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658b39a-e415-46c5-9225-a8519186689d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:07.146986 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658b39a-e415-46c5-9225-a8519186689d admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:07.148155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658b39a-e415-46c5-9225-a8519186689d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:07.151798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658b39a-e415-46c5-9225-a8519186689d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:07.261320 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1321/2591] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:07 2022] POST /identity/v3/projects => generated 401 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:07.265632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91464d2c-837b-44aa-92b9-dc6490bb76bc None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:07.281597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91464d2c-837b-44aa-92b9-dc6490bb76bc None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:07.360634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:07.388393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf78103-2588-48c8-ba13-a375c262fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:07.389256 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78103-2588-48c8-ba13-a375c262fea6 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:07.389687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78103-2588-48c8-ba13-a375c262fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:07.390107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78103-2588-48c8-ba13-a375c262fea6 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:07.391106 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf78103-2588-48c8-ba13-a375c262fea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:07.394493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf78103-2588-48c8-ba13-a375c262fea6 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:07.500928 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1271/2592] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:07 2022] POST /identity/v3/users => generated 307 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:07.506992 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658b39a-e415-46c5-9225-a8519186689d admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:07.523134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658b39a-e415-46c5-9225-a8519186689d admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:07.529523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a191-ee3a-4a39-9a4e-12f62ff7d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:07.530404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a191-ee3a-4a39-9a4e-12f62ff7d3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:07.530894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a191-ee3a-4a39-9a4e-12f62ff7d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:07.531405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a191-ee3a-4a39-9a4e-12f62ff7d3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:07.532402 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465a191-ee3a-4a39-9a4e-12f62ff7d3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:07.534509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465a191-ee3a-4a39-9a4e-12f62ff7d3e0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:07.546853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1322/2593] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:07.554706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf78103-2588-48c8-ba13-a375c262fea6 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:07.565820 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf78103-2588-48c8-ba13-a375c262fea6 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:07.570913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8f0e0-1cc5-4672-acdd-c55d0c7ef877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:07.571618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f0e0-1cc5-4672-acdd-c55d0c7ef877 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:07.571901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f0e0-1cc5-4672-acdd-c55d0c7ef877 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:07.572200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f0e0-1cc5-4672-acdd-c55d0c7ef877 admin admin] PATH_INFO: `/v3/projects/bf2ed75f42d74452a303b1ff2a88496b/users/37a83f500e5c4187a4c6dab84661d4b9/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:07.590750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8f0e0-1cc5-4672-acdd-c55d0c7ef877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2ed75f42d74452a303b1ff2a88496b, user_id=37a83f500e5c4187a4c6dab84661d4b9, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:07.593823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8f0e0-1cc5-4672-acdd-c55d0c7ef877 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:07.689208 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1272/2594] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:07 2022] PUT /identity/v3/projects/bf2ed75f42d74452a303b1ff2a88496b/users/37a83f500e5c4187a4c6dab84661d4b9/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:07.694091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a191-ee3a-4a39-9a4e-12f62ff7d3e0 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:07.705249 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a191-ee3a-4a39-9a4e-12f62ff7d3e0 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:07.773879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a2ef7-afc7-4bac-9149-d8d6428e9845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:07.775084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a2ef7-afc7-4bac-9149-d8d6428e9845 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:07.775422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a2ef7-afc7-4bac-9149-d8d6428e9845 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:07.775848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a2ef7-afc7-4bac-9149-d8d6428e9845 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:07.776968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a2ef7-afc7-4bac-9149-d8d6428e9845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:07.778940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a2ef7-afc7-4bac-9149-d8d6428e9845 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:07.797850 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1323/2595] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:07.803107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8f0e0-1cc5-4672-acdd-c55d0c7ef877 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:07.814919 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8f0e0-1cc5-4672-acdd-c55d0c7ef877 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:07.819673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a452af9-1d6f-44b6-9b16-5d4b9b731f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:07.820887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a452af9-1d6f-44b6-9b16-5d4b9b731f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:07.820887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a452af9-1d6f-44b6-9b16-5d4b9b731f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:07.821637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a452af9-1d6f-44b6-9b16-5d4b9b731f15 admin admin] PATH_INFO: `/v3/projects/bf2ed75f42d74452a303b1ff2a88496b/users/37a83f500e5c4187a4c6dab84661d4b9/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:07.823689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a452af9-1d6f-44b6-9b16-5d4b9b731f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2ed75f42d74452a303b1ff2a88496b, user_id=37a83f500e5c4187a4c6dab84661d4b9, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:07.832960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a452af9-1d6f-44b6-9b16-5d4b9b731f15 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:07.845191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1273/2596] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:07 2022] PUT /identity/v3/projects/bf2ed75f42d74452a303b1ff2a88496b/users/37a83f500e5c4187a4c6dab84661d4b9/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:07.857732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796837fa-8078-4ed5-8287-819c9d831109 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:07.858258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796837fa-8078-4ed5-8287-819c9d831109 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:07.858769 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796837fa-8078-4ed5-8287-819c9d831109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:07.910026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-796837fa-8078-4ed5-8287-819c9d831109 None None] MFA Rules not processed for user `37a83f500e5c4187a4c6dab84661d4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:07.911325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-796837fa-8078-4ed5-8287-819c9d831109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:07.997399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1324/2597] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:07 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:08.009375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a452af9-1d6f-44b6-9b16-5d4b9b731f15 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:08.012614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8a452af9-1d6f-44b6-9b16-5d4b9b731f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:08.073751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a452af9-1d6f-44b6-9b16-5d4b9b731f15 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:08.077670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0d99b-eab9-4a8a-8635-5358c8e6ea58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:08.078337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0d99b-eab9-4a8a-8635-5358c8e6ea58 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:08.078614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0d99b-eab9-4a8a-8635-5358c8e6ea58 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:08.078939 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0d99b-eab9-4a8a-8635-5358c8e6ea58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:08.089699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0d99b-eab9-4a8a-8635-5358c8e6ea58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:08.091158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0d99b-eab9-4a8a-8635-5358c8e6ea58 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:08.102063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1274/2598] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:08 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:08.753538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796837fa-8078-4ed5-8287-819c9d831109 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:08.767542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796837fa-8078-4ed5-8287-819c9d831109 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:08.771689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0526b1d-ce7e-4ca1-94a5-95fee8d671f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:08.772348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0526b1d-ce7e-4ca1-94a5-95fee8d671f6 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:08.772627 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0526b1d-ce7e-4ca1-94a5-95fee8d671f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:08.772899 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0526b1d-ce7e-4ca1-94a5-95fee8d671f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:08.823169 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:08.856325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0526b1d-ce7e-4ca1-94a5-95fee8d671f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:08.857799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0526b1d-ce7e-4ca1-94a5-95fee8d671f6 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:08.868788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1325/2599] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:08 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:12.574930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae22f-2ecc-4072-9663-f5d1a2d6b8c8 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:12.575710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae22f-2ecc-4072-9663-f5d1a2d6b8c8 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:12.575710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae22f-2ecc-4072-9663-f5d1a2d6b8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:12.601130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-1a0ae22f-2ecc-4072-9663-f5d1a2d6b8c8 None None] MFA Rules not processed for user `76771369a0df4c5189e42291ebdf9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:12.603841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-1a0ae22f-2ecc-4072-9663-f5d1a2d6b8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:12.701821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1275/2600] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:13.653294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0526b1d-ce7e-4ca1-94a5-95fee8d671f6 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:13.655100 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e0526b1d-ce7e-4ca1-94a5-95fee8d671f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:13.737352 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0526b1d-ce7e-4ca1-94a5-95fee8d671f6 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:13.742422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:13.743700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:13.743986 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:13.744293 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] PATH_INFO: `/v3/users/dbc89e2bc58949c299c67d671b212fab` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:13.745685 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbc89e2bc58949c299c67d671b212fab)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:13.747645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:13.971188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbc89e2bc58949c299c67d671b212fab'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:13.972031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbc89e2bc58949c299c67d671b212fab'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:13.982147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbc89e2bc58949c299c67d671b212fab'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:13.983291 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbc89e2bc58949c299c67d671b212fab'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:14.093731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbc89e2bc58949c299c67d671b212fab'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:14.117688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1326/2601] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:13 2022] DELETE /identity/v3/users/dbc89e2bc58949c299c67d671b212fab => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:14.122728 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3566d-6cc8-4bf6-a269-f8c630aa3dbd None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:14.123564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3566d-6cc8-4bf6-a269-f8c630aa3dbd None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:14.123564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3566d-6cc8-4bf6-a269-f8c630aa3dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:14.147536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-76c3566d-6cc8-4bf6-a269-f8c630aa3dbd None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:14.150074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-76c3566d-6cc8-4bf6-a269-f8c630aa3dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:14.209450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:14.238791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1276/2602] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:14.250283 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:14.252556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:14.334707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:14.339981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54abfef-1719-4601-b4ab-dc8df9aa7157 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:14.344037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da486a06-a007-45b6-8f14-556514ec2cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:14.344692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da486a06-a007-45b6-8f14-556514ec2cdb service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:14.345088 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da486a06-a007-45b6-8f14-556514ec2cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:14.345326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da486a06-a007-45b6-8f14-556514ec2cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:14.357475 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da486a06-a007-45b6-8f14-556514ec2cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:14.359044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da486a06-a007-45b6-8f14-556514ec2cdb service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:14.371301 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1327/2603] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:14.986915 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3566d-6cc8-4bf6-a269-f8c630aa3dbd None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:14.988945 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-76c3566d-6cc8-4bf6-a269-f8c630aa3dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:15.014761 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3566d-6cc8-4bf6-a269-f8c630aa3dbd None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:15.028452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:15.029372 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:15.029839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:15.030284 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] PATH_INFO: `/v3/projects/7ade57641e144d489592c1b14503000b` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:15.032012 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ade57641e144d489592c1b14503000b)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:15.034236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:15.230615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ade57641e144d489592c1b14503000b', 'request_id': 'req-f040b150-95b6-4c0c-afb8-feade0fe9bcc'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:15.237364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ade57641e144d489592c1b14503000b', 'request_id': 'req-f040b150-95b6-4c0c-afb8-feade0fe9bcc'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:15.245426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ade57641e144d489592c1b14503000b', 'request_id': 'req-f040b150-95b6-4c0c-afb8-feade0fe9bcc'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:15.342063 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] The token cache is being invalidate because project 7ade57641e144d489592c1b14503000b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:15.342527 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:15.343238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:15.344885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1277/2604] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:14 2022] DELETE /identity/v3/projects/7ade57641e144d489592c1b14503000b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:15.349573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da486a06-a007-45b6-8f14-556514ec2cdb service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:15.361467 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da486a06-a007-45b6-8f14-556514ec2cdb service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:15.433439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:15.434229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:15.434671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:15.435029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] PATH_INFO: `/v3/users/53dae47352784ce78222d148a83d05a8` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:15.436414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53dae47352784ce78222d148a83d05a8)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:15.438405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:15.690502 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53dae47352784ce78222d148a83d05a8'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:15.693428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53dae47352784ce78222d148a83d05a8'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:15.705649 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53dae47352784ce78222d148a83d05a8'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:15.706782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53dae47352784ce78222d148a83d05a8'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:15.822836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53dae47352784ce78222d148a83d05a8'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:15.854993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1328/2605] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:15 2022] DELETE /identity/v3/users/53dae47352784ce78222d148a83d05a8 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:16.061316 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:16.062807 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:16.077942 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040b150-95b6-4c0c-afb8-feade0fe9bcc admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:16.155512 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:16.156248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:16.156615 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:16.157014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] PATH_INFO: `/v3/users/9031ef5177d24512b18ad15c36cd6878` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:16.158572 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9031ef5177d24512b18ad15c36cd6878)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:16.160844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:16.441238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9031ef5177d24512b18ad15c36cd6878'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:16.450927 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9031ef5177d24512b18ad15c36cd6878'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:16.451821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9031ef5177d24512b18ad15c36cd6878'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:16.511970 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:16.514691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:16.527854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9031ef5177d24512b18ad15c36cd6878'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:16.534781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:16.538851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f182f0-6d11-4f1f-b167-554a83f187c9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:16.551116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:16.552211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:16.552710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:16.553206 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] PATH_INFO: `/v3/projects/ed34e4d506cc4cb99167bb89a4be5414` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:16.555141 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8389070f-6d31-4fa5-9d9f-a5df6fa34e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9031ef5177d24512b18ad15c36cd6878'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:16.555410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed34e4d506cc4cb99167bb89a4be5414)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:16.558954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:16.562843 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1278/2606] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:16 2022] DELETE /identity/v3/users/9031ef5177d24512b18ad15c36cd6878 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:16.572489 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f386611-f609-499f-ad15-1853f3026284 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:16.573591 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f386611-f609-499f-ad15-1853f3026284 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:16.574075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f386611-f609-499f-ad15-1853f3026284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:16.605975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-2f386611-f609-499f-ad15-1853f3026284 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:16.609226 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2f386611-f609-499f-ad15-1853f3026284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:16.687002 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1279/2607] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:16.692924 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed34e4d506cc4cb99167bb89a4be5414', 'request_id': 'req-673e2366-6f5a-4dfe-8729-3cc1e66e711c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:16.700032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f386611-f609-499f-ad15-1853f3026284 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:16.702595 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2f386611-f609-499f-ad15-1853f3026284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:16.706986 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed34e4d506cc4cb99167bb89a4be5414', 'request_id': 'req-673e2366-6f5a-4dfe-8729-3cc1e66e711c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:16.796500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f386611-f609-499f-ad15-1853f3026284 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:16.832526 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed34e4d506cc4cb99167bb89a4be5414', 'request_id': 'req-673e2366-6f5a-4dfe-8729-3cc1e66e711c'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:16.834251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:16.858470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17414d-41d8-489e-944e-e3d6748aa280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:16.859212 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17414d-41d8-489e-944e-e3d6748aa280 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:16.859496 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17414d-41d8-489e-944e-e3d6748aa280 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:16.859771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17414d-41d8-489e-944e-e3d6748aa280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:16.868604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] The token cache is being invalidate because project ed34e4d506cc4cb99167bb89a4be5414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:16.869101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:16.870749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-673e2366-6f5a-4dfe-8729-3cc1e66e711c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:16.872929 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1329/2608] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:16 2022] DELETE /identity/v3/projects/ed34e4d506cc4cb99167bb89a4be5414 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:16.879001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4714b8-059b-48d7-a4c3-76e66959b9cb None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:16.882409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4714b8-059b-48d7-a4c3-76e66959b9cb None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:16.882409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4714b8-059b-48d7-a4c3-76e66959b9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:16.902789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-aa4714b8-059b-48d7-a4c3-76e66959b9cb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:16.904501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-aa4714b8-059b-48d7-a4c3-76e66959b9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:16.935840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:16.955147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17414d-41d8-489e-944e-e3d6748aa280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:16.960767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17414d-41d8-489e-944e-e3d6748aa280 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:16.977082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1280/2609] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:17.009198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1330/2610] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:17.022629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c0a6f-579e-46aa-83eb-a46539fd95e3 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:17.023873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c0a6f-579e-46aa-83eb-a46539fd95e3 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:17.023873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c0a6f-579e-46aa-83eb-a46539fd95e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:17.059198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a79c0a6f-579e-46aa-83eb-a46539fd95e3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:17.061434 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a79c0a6f-579e-46aa-83eb-a46539fd95e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:17.076491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1281/2611] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:17.083229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4714b8-059b-48d7-a4c3-76e66959b9cb None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:17.103845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4714b8-059b-48d7-a4c3-76e66959b9cb None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:17.114536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ef2e-4929-4011-b87d-3b93e0bd05cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:17.115692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ef2e-4929-4011-b87d-3b93e0bd05cd admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:17.116230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ef2e-4929-4011-b87d-3b93e0bd05cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:17.116742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ef2e-4929-4011-b87d-3b93e0bd05cd admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:17.118448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ef2e-4929-4011-b87d-3b93e0bd05cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:12:17.119003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ef2e-4929-4011-b87d-3b93e0bd05cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:17.122218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ef2e-4929-4011-b87d-3b93e0bd05cd admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:17.147421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1331/2612] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:12:17.149380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c0a6f-579e-46aa-83eb-a46539fd95e3 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:17.159166 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c0a6f-579e-46aa-83eb-a46539fd95e3 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:17.165521 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52658ea-28c1-4588-87d6-f1f9727fba43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:17.166299 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52658ea-28c1-4588-87d6-f1f9727fba43 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:17.166697 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52658ea-28c1-4588-87d6-f1f9727fba43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:17.167091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52658ea-28c1-4588-87d6-f1f9727fba43 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:17.168072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52658ea-28c1-4588-87d6-f1f9727fba43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:17.170377 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52658ea-28c1-4588-87d6-f1f9727fba43 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:17.450686 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1282/2613] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:17 2022] POST /identity/v3/projects => generated 367 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:17.454815 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ef2e-4929-4011-b87d-3b93e0bd05cd admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:17.471573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ef2e-4929-4011-b87d-3b93e0bd05cd admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:17.580843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c3619-10b7-41ad-99fe-ff273afe0df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:17.582972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c3619-10b7-41ad-99fe-ff273afe0df8 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:17.583900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c3619-10b7-41ad-99fe-ff273afe0df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:17.584659 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c3619-10b7-41ad-99fe-ff273afe0df8 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:17.586878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c3619-10b7-41ad-99fe-ff273afe0df8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:17.591886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c3619-10b7-41ad-99fe-ff273afe0df8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:17.683042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1332/2614] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:17 2022] POST /identity/v3/users => generated 284 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:17.684051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52658ea-28c1-4588-87d6-f1f9727fba43 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:17.686420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c3619-10b7-41ad-99fe-ff273afe0df8 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:17.686610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b52658ea-28c1-4588-87d6-f1f9727fba43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:17.702304 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52658ea-28c1-4588-87d6-f1f9727fba43 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:17.709804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c3619-10b7-41ad-99fe-ff273afe0df8 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:17.710144 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:17.711773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:17.712423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:17.713103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] PATH_INFO: `/v3/projects/0b9e841054e045c29a3fef523a8e79e0` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:17.715530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9e841054e045c29a3fef523a8e79e0)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:17.718755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:17.721829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eeea8-0d6c-4c0e-a2ae-815455e8df83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:17.723115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eeea8-0d6c-4c0e-a2ae-815455e8df83 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:17.723798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eeea8-0d6c-4c0e-a2ae-815455e8df83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:17.724146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eeea8-0d6c-4c0e-a2ae-815455e8df83 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:17.730221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9eeea8-0d6c-4c0e-a2ae-815455e8df83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:17.730221 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9eeea8-0d6c-4c0e-a2ae-815455e8df83 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:17.745656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1333/2615] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:17.751281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eeea8-0d6c-4c0e-a2ae-815455e8df83 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:17.764035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eeea8-0d6c-4c0e-a2ae-815455e8df83 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:17.769209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6577ab-3f7d-4773-bf20-df25b7a101e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:17.769870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6577ab-3f7d-4773-bf20-df25b7a101e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:17.770157 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6577ab-3f7d-4773-bf20-df25b7a101e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:17.770432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6577ab-3f7d-4773-bf20-df25b7a101e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:17.771094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6577ab-3f7d-4773-bf20-df25b7a101e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:17.772878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6577ab-3f7d-4773-bf20-df25b7a101e0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:17.787428 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1334/2616] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:17.793326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6577ab-3f7d-4773-bf20-df25b7a101e0 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:17.808302 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6577ab-3f7d-4773-bf20-df25b7a101e0 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:17.813871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eca7c5-ff79-4c88-a15b-de330811e6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:17.814730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eca7c5-ff79-4c88-a15b-de330811e6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:17.815187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eca7c5-ff79-4c88-a15b-de330811e6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:17.815587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eca7c5-ff79-4c88-a15b-de330811e6b4 admin admin] PATH_INFO: `/v3/projects/175810adfe8d4763aab48a40692ace96/users/e7a25233ba9d43d69a30241e3e3648ca/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:17.834778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eca7c5-ff79-4c88-a15b-de330811e6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175810adfe8d4763aab48a40692ace96, user_id=e7a25233ba9d43d69a30241e3e3648ca, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:17.837607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eca7c5-ff79-4c88-a15b-de330811e6b4 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:17.960386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1335/2617] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:17 2022] PUT /identity/v3/projects/175810adfe8d4763aab48a40692ace96/users/e7a25233ba9d43d69a30241e3e3648ca/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:17.972277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87aa081-ceeb-45f2-b2e5-efc2c563ab04 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:17.979836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87aa081-ceeb-45f2-b2e5-efc2c563ab04 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:17.979836 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87aa081-ceeb-45f2-b2e5-efc2c563ab04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:17.980313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9e841054e045c29a3fef523a8e79e0', 'request_id': 'req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:17.983136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9e841054e045c29a3fef523a8e79e0', 'request_id': 'req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:17.992398 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9e841054e045c29a3fef523a8e79e0', 'request_id': 'req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:18.009646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f87aa081-ceeb-45f2-b2e5-efc2c563ab04 None None] MFA Rules not processed for user `e7a25233ba9d43d69a30241e3e3648ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:18.010969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f87aa081-ceeb-45f2-b2e5-efc2c563ab04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:18.029213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:18.091099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1336/2618] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:18.095882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87aa081-ceeb-45f2-b2e5-efc2c563ab04 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:18.109155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] The token cache is being invalidate because project 0b9e841054e045c29a3fef523a8e79e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:18.109743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:18.110783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:18.113352 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1283/2619] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:17 2022] DELETE /identity/v3/projects/0b9e841054e045c29a3fef523a8e79e0 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:18.119664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:18.122097 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:18.194431 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87aa081-ceeb-45f2-b2e5-efc2c563ab04 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:18.203700 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eec1d-cac2-4c18-b8d5-e09cb527f446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:18.204811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eec1d-cac2-4c18-b8d5-e09cb527f446 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:18.205404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eec1d-cac2-4c18-b8d5-e09cb527f446 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:18.205883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eec1d-cac2-4c18-b8d5-e09cb527f446 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:18.207109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eec1d-cac2-4c18-b8d5-e09cb527f446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:18.210813 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eec1d-cac2-4c18-b8d5-e09cb527f446 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:18.216803 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867aa8c-13c7-4a2a-b2f8-3ddb1ebd5f14 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:18.221943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:18.229529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:18.229529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:18.229529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] PATH_INFO: `/v3/users/37a83f500e5c4187a4c6dab84661d4b9` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:18.229529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a83f500e5c4187a4c6dab84661d4b9)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:18.229529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:18.372508 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1337/2620] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:18 2022] POST /identity/v3/projects => generated 365 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:18.378043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eec1d-cac2-4c18-b8d5-e09cb527f446 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:18.380457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3a9eec1d-cac2-4c18-b8d5-e09cb527f446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:18.459076 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eec1d-cac2-4c18-b8d5-e09cb527f446 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:18.465228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7adf5-24f8-4822-9162-f522c4cef2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:18.466146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7adf5-24f8-4822-9162-f522c4cef2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:18.466574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7adf5-24f8-4822-9162-f522c4cef2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:18.466960 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7adf5-24f8-4822-9162-f522c4cef2f6 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:18.467714 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7adf5-24f8-4822-9162-f522c4cef2f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:18.470830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7adf5-24f8-4822-9162-f522c4cef2f6 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:18.491523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a83f500e5c4187a4c6dab84661d4b9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:18.493801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:18.505908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a83f500e5c4187a4c6dab84661d4b9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:18.507110 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a83f500e5c4187a4c6dab84661d4b9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:18.618070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1338/2621] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:18 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:18.619718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7adf5-24f8-4822-9162-f522c4cef2f6 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:18.622621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e8d7adf5-24f8-4822-9162-f522c4cef2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:18.647420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7adf5-24f8-4822-9162-f522c4cef2f6 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:18.657312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e52e7-86c5-46d4-b407-cf59113bf029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:18.658398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e52e7-86c5-46d4-b407-cf59113bf029 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:18.658803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e52e7-86c5-46d4-b407-cf59113bf029 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:18.659261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e52e7-86c5-46d4-b407-cf59113bf029 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:18.660493 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e52e7-86c5-46d4-b407-cf59113bf029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:18.663560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e52e7-86c5-46d4-b407-cf59113bf029 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:18.675073 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a83f500e5c4187a4c6dab84661d4b9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:18.692730 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1339/2622] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:18.695315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e52e7-86c5-46d4-b407-cf59113bf029 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:18.703777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a83f500e5c4187a4c6dab84661d4b9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:18.717084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1284/2623] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:18 2022] DELETE /identity/v3/users/37a83f500e5c4187a4c6dab84661d4b9 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:18.727772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e52e7-86c5-46d4-b407-cf59113bf029 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:18.750230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:18.840871 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd759b-0cfa-41e9-8753-fdb60f135b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:18.842033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd759b-0cfa-41e9-8753-fdb60f135b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:18.842495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd759b-0cfa-41e9-8753-fdb60f135b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:18.843257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd759b-0cfa-41e9-8753-fdb60f135b25 admin admin] PATH_INFO: `/v3/projects/d9a7e1bb0fb64c94ace1baf7242042fe/users/33cf9829e5bf49b7a0d33adf2732b834/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:18.869735 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd759b-0cfa-41e9-8753-fdb60f135b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a7e1bb0fb64c94ace1baf7242042fe, user_id=33cf9829e5bf49b7a0d33adf2732b834, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:18.875343 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd759b-0cfa-41e9-8753-fdb60f135b25 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:19.082633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1340/2624] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:18 2022] PUT /identity/v3/projects/d9a7e1bb0fb64c94ace1baf7242042fe/users/33cf9829e5bf49b7a0d33adf2732b834/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:19.089672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:19.107134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58d634-03eb-42e1-93bd-749470507590 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:19.207406 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c490131-c595-4a32-ae1d-5edad4aed28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:19.208202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c490131-c595-4a32-ae1d-5edad4aed28a admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:19.208650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c490131-c595-4a32-ae1d-5edad4aed28a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:19.215552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c490131-c595-4a32-ae1d-5edad4aed28a admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:19.215552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c490131-c595-4a32-ae1d-5edad4aed28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:19.215552 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c490131-c595-4a32-ae1d-5edad4aed28a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:19.226508 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1285/2625] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:19.232404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd759b-0cfa-41e9-8753-fdb60f135b25 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:19.249766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd759b-0cfa-41e9-8753-fdb60f135b25 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:19.257684 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d18b9-e448-4231-b582-0d43c0da6e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:19.258768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d18b9-e448-4231-b582-0d43c0da6e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:19.259207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d18b9-e448-4231-b582-0d43c0da6e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:19.259646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d18b9-e448-4231-b582-0d43c0da6e0e admin admin] PATH_INFO: `/v3/projects/d9a7e1bb0fb64c94ace1baf7242042fe/users/33cf9829e5bf49b7a0d33adf2732b834/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:19.263029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d18b9-e448-4231-b582-0d43c0da6e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a7e1bb0fb64c94ace1baf7242042fe, user_id=33cf9829e5bf49b7a0d33adf2732b834, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:19.267785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d18b9-e448-4231-b582-0d43c0da6e0e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:19.295225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1341/2626] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:19 2022] PUT /identity/v3/projects/d9a7e1bb0fb64c94ace1baf7242042fe/users/33cf9829e5bf49b7a0d33adf2732b834/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:19.312792 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257c6d4-93aa-4441-9983-f0181378ad8f None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:19.313587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257c6d4-93aa-4441-9983-f0181378ad8f None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:19.314234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257c6d4-93aa-4441-9983-f0181378ad8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:19.363720 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7257c6d4-93aa-4441-9983-f0181378ad8f None None] MFA Rules not processed for user `33cf9829e5bf49b7a0d33adf2732b834`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:19.365792 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7257c6d4-93aa-4441-9983-f0181378ad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:19.473519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d18b9-e448-4231-b582-0d43c0da6e0e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:19.479087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1286/2627] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:19.479319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3d2d18b9-e448-4231-b582-0d43c0da6e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:19.492977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257c6d4-93aa-4441-9983-f0181378ad8f None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:19.495678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7257c6d4-93aa-4441-9983-f0181378ad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:19.579350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:19.586989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257c6d4-93aa-4441-9983-f0181378ad8f None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:19.591168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f2d34-86c3-4c2b-84b0-bf9f094dd937 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:19.591794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f2d34-86c3-4c2b-84b0-bf9f094dd937 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:19.592074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f2d34-86c3-4c2b-84b0-bf9f094dd937 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:19.592382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f2d34-86c3-4c2b-84b0-bf9f094dd937 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:19.600315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d18b9-e448-4231-b582-0d43c0da6e0e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:19.603785 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f2d34-86c3-4c2b-84b0-bf9f094dd937 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:19.605324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f2d34-86c3-4c2b-84b0-bf9f094dd937 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:19.609491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:19.610608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:19.611100 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:19.611556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] PATH_INFO: `/v3/projects/bf2ed75f42d74452a303b1ff2a88496b` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:19.613916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf2ed75f42d74452a303b1ff2a88496b)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:19.617334 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1287/2628] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:19.618443 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:19.779609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf2ed75f42d74452a303b1ff2a88496b', 'request_id': 'req-77f3ae85-727b-4db7-8668-7f852dd36928'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:19.786922 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf2ed75f42d74452a303b1ff2a88496b', 'request_id': 'req-77f3ae85-727b-4db7-8668-7f852dd36928'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:19.882886 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f2d34-86c3-4c2b-84b0-bf9f094dd937 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:19.892949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f2d34-86c3-4c2b-84b0-bf9f094dd937 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:19.931650 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf2ed75f42d74452a303b1ff2a88496b', 'request_id': 'req-77f3ae85-727b-4db7-8668-7f852dd36928'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:19.949393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] The token cache is being invalidate because project bf2ed75f42d74452a303b1ff2a88496b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:19.949970 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:19.950989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-77f3ae85-727b-4db7-8668-7f852dd36928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:19.954107 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1342/2629] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:19 2022] DELETE /identity/v3/projects/bf2ed75f42d74452a303b1ff2a88496b => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:19.959393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66226884-8c85-4006-a0dd-4a5d9dd7adce None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:19.959914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66226884-8c85-4006-a0dd-4a5d9dd7adce None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:19.960782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66226884-8c85-4006-a0dd-4a5d9dd7adce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:19.970170 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975b46b-ab1b-4efb-ad76-62ef830f5a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:19.970788 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975b46b-ab1b-4efb-ad76-62ef830f5a37 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:19.971066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975b46b-ab1b-4efb-ad76-62ef830f5a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:19.971339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975b46b-ab1b-4efb-ad76-62ef830f5a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:19.973582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2975b46b-ab1b-4efb-ad76-62ef830f5a37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:19.988990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-66226884-8c85-4006-a0dd-4a5d9dd7adce None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:19.990246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-66226884-8c85-4006-a0dd-4a5d9dd7adce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:20.010543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:20.082361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975b46b-ab1b-4efb-ad76-62ef830f5a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:20.083858 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975b46b-ab1b-4efb-ad76-62ef830f5a37 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:20.096154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1288/2630] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:20.124803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1343/2631] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:20.133317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed2506-4404-4fdf-9a5f-9ccee8ff0606 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:20.134162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed2506-4404-4fdf-9a5f-9ccee8ff0606 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:20.134618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed2506-4404-4fdf-9a5f-9ccee8ff0606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:20.181515 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-59ed2506-4404-4fdf-9a5f-9ccee8ff0606 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:20.183770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-59ed2506-4404-4fdf-9a5f-9ccee8ff0606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:20.198382 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1289/2632] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:20.205665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66226884-8c85-4006-a0dd-4a5d9dd7adce None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:20.235040 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66226884-8c85-4006-a0dd-4a5d9dd7adce None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:20.244832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf84905-0c10-41ff-b91c-ceb2b185817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:20.244832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf84905-0c10-41ff-b91c-ceb2b185817e admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:20.244832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf84905-0c10-41ff-b91c-ceb2b185817e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:20.244832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf84905-0c10-41ff-b91c-ceb2b185817e admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:20.245598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf84905-0c10-41ff-b91c-ceb2b185817e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:12:20.245598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf84905-0c10-41ff-b91c-ceb2b185817e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:20.246033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf84905-0c10-41ff-b91c-ceb2b185817e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:20.272369 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed2506-4404-4fdf-9a5f-9ccee8ff0606 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:20.272868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1344/2633] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:12:20.297749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed2506-4404-4fdf-9a5f-9ccee8ff0606 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:20.297749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26861-087f-4a8c-85c8-00024b298a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:20.297749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26861-087f-4a8c-85c8-00024b298a49 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:20.297749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26861-087f-4a8c-85c8-00024b298a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:20.298490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26861-087f-4a8c-85c8-00024b298a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:20.298490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d26861-087f-4a8c-85c8-00024b298a49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:20.298490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d26861-087f-4a8c-85c8-00024b298a49 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:20.457805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1290/2634] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:20 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:20.463985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf84905-0c10-41ff-b91c-ceb2b185817e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:20.475963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf84905-0c10-41ff-b91c-ceb2b185817e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:20.558316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a5d6d-b30c-41b7-bf5b-c22d159234e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:20.558946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a5d6d-b30c-41b7-bf5b-c22d159234e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:20.559230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a5d6d-b30c-41b7-bf5b-c22d159234e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:20.559509 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a5d6d-b30c-41b7-bf5b-c22d159234e8 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:20.560247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a5d6d-b30c-41b7-bf5b-c22d159234e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:20.562252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a5d6d-b30c-41b7-bf5b-c22d159234e8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:20.717363 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1345/2635] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:20 2022] POST /identity/v3/users => generated 292 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:20.723767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26861-087f-4a8c-85c8-00024b298a49 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:20.747006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26861-087f-4a8c-85c8-00024b298a49 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:20.755494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f44fe-9b48-4488-8270-b20d224ec3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:20.756589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f44fe-9b48-4488-8270-b20d224ec3bf admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:20.757067 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f44fe-9b48-4488-8270-b20d224ec3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:20.757573 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f44fe-9b48-4488-8270-b20d224ec3bf admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:20.758714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f44fe-9b48-4488-8270-b20d224ec3bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:20.761916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f44fe-9b48-4488-8270-b20d224ec3bf admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:20.779564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1291/2636] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:20.784833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a5d6d-b30c-41b7-bf5b-c22d159234e8 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:20.800642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a5d6d-b30c-41b7-bf5b-c22d159234e8 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:20.806718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba761f0f-66bc-4e4b-b6b7-767415cb6272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:20.807525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761f0f-66bc-4e4b-b6b7-767415cb6272 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:20.807916 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761f0f-66bc-4e4b-b6b7-767415cb6272 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:20.808357 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761f0f-66bc-4e4b-b6b7-767415cb6272 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:20.809579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba761f0f-66bc-4e4b-b6b7-767415cb6272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:20.811655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba761f0f-66bc-4e4b-b6b7-767415cb6272 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:20.825841 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1346/2637] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:20.832969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f44fe-9b48-4488-8270-b20d224ec3bf admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:20.850404 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f44fe-9b48-4488-8270-b20d224ec3bf admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:20.859051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdfe00-d819-4547-8e23-bc9be7c34813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:20.860240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdfe00-d819-4547-8e23-bc9be7c34813 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:20.867561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdfe00-d819-4547-8e23-bc9be7c34813 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:20.867561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdfe00-d819-4547-8e23-bc9be7c34813 admin admin] PATH_INFO: `/v3/projects/6e65cf18a7da47f4a61c47ee2dccea8e/users/ce7c4186359b46078a75c4dc260ca50d/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:20.885791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdfe00-d819-4547-8e23-bc9be7c34813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e65cf18a7da47f4a61c47ee2dccea8e, user_id=ce7c4186359b46078a75c4dc260ca50d, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:20.891390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdfe00-d819-4547-8e23-bc9be7c34813 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:20.993477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1292/2638] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:20 2022] PUT /identity/v3/projects/6e65cf18a7da47f4a61c47ee2dccea8e/users/ce7c4186359b46078a75c4dc260ca50d/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:21.006074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be1fa-d8a9-4043-bb7c-f296747bccac None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:21.006469 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be1fa-d8a9-4043-bb7c-f296747bccac None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:21.006796 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be1fa-d8a9-4043-bb7c-f296747bccac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:21.053318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-5d6be1fa-d8a9-4043-bb7c-f296747bccac None None] MFA Rules not processed for user `ce7c4186359b46078a75c4dc260ca50d`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:21.055434 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5d6be1fa-d8a9-4043-bb7c-f296747bccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:21.144000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1347/2639] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:21.149577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdfe00-d819-4547-8e23-bc9be7c34813 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:21.220145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:21.259709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdfe00-d819-4547-8e23-bc9be7c34813 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:21.268300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1240d-cb52-4f65-b3d9-a79a917c3b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:21.269779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1240d-cb52-4f65-b3d9-a79a917c3b6d admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:21.270070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1240d-cb52-4f65-b3d9-a79a917c3b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:21.270676 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1240d-cb52-4f65-b3d9-a79a917c3b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:21.271972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1240d-cb52-4f65-b3d9-a79a917c3b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:21.276223 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1240d-cb52-4f65-b3d9-a79a917c3b6d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:21.407744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1293/2640] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:21 2022] POST /identity/v3/projects => generated 381 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:21.408784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6be1fa-d8a9-4043-bb7c-f296747bccac None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:21.427639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6be1fa-d8a9-4043-bb7c-f296747bccac None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:21.445513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:21.524927 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cf1d2-2b55-4175-bfb3-61aecb0cca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:21.526152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cf1d2-2b55-4175-bfb3-61aecb0cca0b admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:21.526654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cf1d2-2b55-4175-bfb3-61aecb0cca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:21.527149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cf1d2-2b55-4175-bfb3-61aecb0cca0b admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:21.528406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cf1d2-2b55-4175-bfb3-61aecb0cca0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:21.532152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cf1d2-2b55-4175-bfb3-61aecb0cca0b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:21.687585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1348/2641] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:21 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:21.692050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1240d-cb52-4f65-b3d9-a79a917c3b6d admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:21.705922 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1240d-cb52-4f65-b3d9-a79a917c3b6d admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:21.714148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abe0dc-0e97-47e7-b2a1-caff802d95e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:21.715167 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abe0dc-0e97-47e7-b2a1-caff802d95e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:21.715629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abe0dc-0e97-47e7-b2a1-caff802d95e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:21.716236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abe0dc-0e97-47e7-b2a1-caff802d95e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:21.717332 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abe0dc-0e97-47e7-b2a1-caff802d95e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:21.720454 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abe0dc-0e97-47e7-b2a1-caff802d95e7 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:21.741168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1294/2642] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:21.746654 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cf1d2-2b55-4175-bfb3-61aecb0cca0b admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:21.758477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cf1d2-2b55-4175-bfb3-61aecb0cca0b admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:21.763942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f69216-9365-485d-88ae-22c7946ff2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:21.765038 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f69216-9365-485d-88ae-22c7946ff2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:21.765426 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f69216-9365-485d-88ae-22c7946ff2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:21.765753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f69216-9365-485d-88ae-22c7946ff2c4 admin admin] PATH_INFO: `/v3/projects/d1a3e973af424e2d9fdd6dea88f8ac97/users/ad9d12a802fe4cb3bc9113cf0cb72e5e/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:21.783268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f69216-9365-485d-88ae-22c7946ff2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a3e973af424e2d9fdd6dea88f8ac97, user_id=ad9d12a802fe4cb3bc9113cf0cb72e5e, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:21.786372 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f69216-9365-485d-88ae-22c7946ff2c4 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:21.896645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1349/2643] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:21 2022] PUT /identity/v3/projects/d1a3e973af424e2d9fdd6dea88f8ac97/users/ad9d12a802fe4cb3bc9113cf0cb72e5e/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:21.901071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abe0dc-0e97-47e7-b2a1-caff802d95e7 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:21.912149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abe0dc-0e97-47e7-b2a1-caff802d95e7 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:21.978738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb0f9c-c689-4de4-98bd-a8f9736d0697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:21.979354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb0f9c-c689-4de4-98bd-a8f9736d0697 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:21.979638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb0f9c-c689-4de4-98bd-a8f9736d0697 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:21.979916 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb0f9c-c689-4de4-98bd-a8f9736d0697 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:21.980612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb0f9c-c689-4de4-98bd-a8f9736d0697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:21.982242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb0f9c-c689-4de4-98bd-a8f9736d0697 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:21.994265 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1295/2644] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:21.998891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f69216-9365-485d-88ae-22c7946ff2c4 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:22.015498 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f69216-9365-485d-88ae-22c7946ff2c4 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:22.025556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2272d46-3928-4eaa-8597-e610320f0d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:22.027041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2272d46-3928-4eaa-8597-e610320f0d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:22.027590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2272d46-3928-4eaa-8597-e610320f0d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:22.028085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2272d46-3928-4eaa-8597-e610320f0d98 admin admin] PATH_INFO: `/v3/projects/d1a3e973af424e2d9fdd6dea88f8ac97/users/ad9d12a802fe4cb3bc9113cf0cb72e5e/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:22.032208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2272d46-3928-4eaa-8597-e610320f0d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a3e973af424e2d9fdd6dea88f8ac97, user_id=ad9d12a802fe4cb3bc9113cf0cb72e5e, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:22.046350 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2272d46-3928-4eaa-8597-e610320f0d98 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:22.059892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1350/2645] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:21 2022] PUT /identity/v3/projects/d1a3e973af424e2d9fdd6dea88f8ac97/users/ad9d12a802fe4cb3bc9113cf0cb72e5e/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:22.071467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8e038-d77e-41ec-bccc-8528dd5c83da None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:22.071755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8e038-d77e-41ec-bccc-8528dd5c83da None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:22.072041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8e038-d77e-41ec-bccc-8528dd5c83da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:22.232323 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-26d8e038-d77e-41ec-bccc-8528dd5c83da None None] MFA Rules not processed for user `ad9d12a802fe4cb3bc9113cf0cb72e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:22.233604 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-26d8e038-d77e-41ec-bccc-8528dd5c83da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:22.317763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1296/2646] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:22.329483 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2272d46-3928-4eaa-8597-e610320f0d98 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:22.331378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c2272d46-3928-4eaa-8597-e610320f0d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:22.409309 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2272d46-3928-4eaa-8597-e610320f0d98 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:22.413143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6ce1f-0bc1-47ec-92a4-63cb98881fff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:22.413781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6ce1f-0bc1-47ec-92a4-63cb98881fff service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:22.414607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6ce1f-0bc1-47ec-92a4-63cb98881fff service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:22.414900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6ce1f-0bc1-47ec-92a4-63cb98881fff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:22.489313 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6ce1f-0bc1-47ec-92a4-63cb98881fff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:22.490809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6ce1f-0bc1-47ec-92a4-63cb98881fff service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:22.502562 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1351/2647] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:25.361938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93529bb8-ea20-45ba-9c13-16e2ebf1d133 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:25.361938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93529bb8-ea20-45ba-9c13-16e2ebf1d133 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:25.365874 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93529bb8-ea20-45ba-9c13-16e2ebf1d133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:25.384744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-93529bb8-ea20-45ba-9c13-16e2ebf1d133 None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:25.386314 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-93529bb8-ea20-45ba-9c13-16e2ebf1d133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:25.426693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:25.454797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1297/2648] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:12:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:25.704911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6ce1f-0bc1-47ec-92a4-63cb98881fff service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:25.706642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-24c6ce1f-0bc1-47ec-92a4-63cb98881fff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:25.786893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:25.883629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6ce1f-0bc1-47ec-92a4-63cb98881fff service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:25.891858 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f11812-4d6f-404c-ac97-63036387b089 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:25.892930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f11812-4d6f-404c-ac97-63036387b089 service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:25.893422 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f11812-4d6f-404c-ac97-63036387b089 service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:25.894010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f11812-4d6f-404c-ac97-63036387b089 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:25.913026 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f11812-4d6f-404c-ac97-63036387b089 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:25.914951 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f11812-4d6f-404c-ac97-63036387b089 service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:25.931401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1352/2649] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:12:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:12:28.517326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93529bb8-ea20-45ba-9c13-16e2ebf1d133 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:28.519440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-93529bb8-ea20-45ba-9c13-16e2ebf1d133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:28.659113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93529bb8-ea20-45ba-9c13-16e2ebf1d133 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:28.666188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:28.666847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:28.667300 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:28.670728 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] PATH_INFO: `/v3/users/e7a25233ba9d43d69a30241e3e3648ca` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:28.671197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7a25233ba9d43d69a30241e3e3648ca)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:28.673449 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:28.963580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7a25233ba9d43d69a30241e3e3648ca'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:28.975019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7a25233ba9d43d69a30241e3e3648ca'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:28.976064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7a25233ba9d43d69a30241e3e3648ca'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:29.098752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7a25233ba9d43d69a30241e3e3648ca'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:29.125046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7a25233ba9d43d69a30241e3e3648ca'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:29.143562 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1298/2650] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:28 2022] DELETE /identity/v3/users/e7a25233ba9d43d69a30241e3e3648ca => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:29.151374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c4192-0ff9-4756-a2b5-3749ad453652 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:29.151942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c4192-0ff9-4756-a2b5-3749ad453652 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:29.151942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c4192-0ff9-4756-a2b5-3749ad453652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:29.188649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:29.189286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-3b8c4192-0ff9-4756-a2b5-3749ad453652 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:29.190149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:29.190814 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3b8c4192-0ff9-4756-a2b5-3749ad453652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:29.265237 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00385240-389e-41a9-852b-76c29de4f54a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:29.269162 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a627acc-49c9-4124-ac42-aa7db15d4b80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:29.269854 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a627acc-49c9-4124-ac42-aa7db15d4b80 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:29.270131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a627acc-49c9-4124-ac42-aa7db15d4b80 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:29.270403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a627acc-49c9-4124-ac42-aa7db15d4b80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:29.272558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3a627acc-49c9-4124-ac42-aa7db15d4b80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:29.289693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:29.377497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a627acc-49c9-4124-ac42-aa7db15d4b80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:29.380664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a627acc-49c9-4124-ac42-aa7db15d4b80 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:29.399587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1299/2651] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:29.426317 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1353/2652] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:29.437240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a627acc-49c9-4124-ac42-aa7db15d4b80 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:29.438770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3a627acc-49c9-4124-ac42-aa7db15d4b80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:29.517455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a627acc-49c9-4124-ac42-aa7db15d4b80 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:29.523195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d1fcf-6d66-4e2c-ba94-854cc1e095b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:29.524200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d1fcf-6d66-4e2c-ba94-854cc1e095b6 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:29.524694 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d1fcf-6d66-4e2c-ba94-854cc1e095b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:29.525750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d1fcf-6d66-4e2c-ba94-854cc1e095b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:29.541084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d1fcf-6d66-4e2c-ba94-854cc1e095b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:29.542579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d1fcf-6d66-4e2c-ba94-854cc1e095b6 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:29.553689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1300/2653] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:30.157084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c4192-0ff9-4756-a2b5-3749ad453652 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:30.159742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3b8c4192-0ff9-4756-a2b5-3749ad453652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:30.190093 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c4192-0ff9-4756-a2b5-3749ad453652 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:30.197043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:30.197587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:30.197892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:30.198213 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] PATH_INFO: `/v3/projects/175810adfe8d4763aab48a40692ace96` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:30.204303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=175810adfe8d4763aab48a40692ace96)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:30.207831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:30.471556 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175810adfe8d4763aab48a40692ace96', 'request_id': 'req-4fe120ea-2055-4266-ad97-e424627e2ac4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:30.498336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175810adfe8d4763aab48a40692ace96', 'request_id': 'req-4fe120ea-2055-4266-ad97-e424627e2ac4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:30.679147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175810adfe8d4763aab48a40692ace96', 'request_id': 'req-4fe120ea-2055-4266-ad97-e424627e2ac4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:30.711748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] The token cache is being invalidate because project 175810adfe8d4763aab48a40692ace96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:30.712791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:30.714472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:30.719132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1354/2654] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:30 2022] DELETE /identity/v3/projects/175810adfe8d4763aab48a40692ace96 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:30.724132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d1fcf-6d66-4e2c-ba94-854cc1e095b6 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:30.757503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d1fcf-6d66-4e2c-ba94-854cc1e095b6 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:30.878472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:30.879715 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:30.880161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:30.880638 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] PATH_INFO: `/v3/users/33cf9829e5bf49b7a0d33adf2732b834` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:30.882030 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33cf9829e5bf49b7a0d33adf2732b834)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:30.885079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:31.189120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33cf9829e5bf49b7a0d33adf2732b834'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:31.198075 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:31.201425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33cf9829e5bf49b7a0d33adf2732b834'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:31.202908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33cf9829e5bf49b7a0d33adf2732b834'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:31.328949 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33cf9829e5bf49b7a0d33adf2732b834'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:31.353739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33cf9829e5bf49b7a0d33adf2732b834'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:31.363873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1301/2655] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:30 2022] DELETE /identity/v3/users/33cf9829e5bf49b7a0d33adf2732b834 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:32.156909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:32.159223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:32.199568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:32.200365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:32.283218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:32.341433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe120ea-2055-4266-ad97-e424627e2ac4 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:32.424014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed017fc-7763-4b21-857d-8dd3129196b5 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:32.432401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b8590-b824-4fa5-b113-76320650bc1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:32.433312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b8590-b824-4fa5-b113-76320650bc1c service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:32.433664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b8590-b824-4fa5-b113-76320650bc1c service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:32.434011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b8590-b824-4fa5-b113-76320650bc1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:32.434539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:32.436020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:32.436161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a35b8590-b824-4fa5-b113-76320650bc1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:32.436590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:32.437514 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] PATH_INFO: `/v3/projects/d9a7e1bb0fb64c94ace1baf7242042fe` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:32.439725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a7e1bb0fb64c94ace1baf7242042fe)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:32.464061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:32.611791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b8590-b824-4fa5-b113-76320650bc1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:32.614356 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b8590-b824-4fa5-b113-76320650bc1c service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:32.641970 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1355/2656] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:32.690867 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a7e1bb0fb64c94ace1baf7242042fe', 'request_id': 'req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:32.702888 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a7e1bb0fb64c94ace1baf7242042fe', 'request_id': 'req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:32.723292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a7e1bb0fb64c94ace1baf7242042fe', 'request_id': 'req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:32.826005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] The token cache is being invalidate because project d9a7e1bb0fb64c94ace1baf7242042fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:32.826291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:32.827045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d208f3ba-2cf1-41b9-b0ba-1f2dd230f33a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:32.840779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1302/2657] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:32 2022] DELETE /identity/v3/projects/d9a7e1bb0fb64c94ace1baf7242042fe => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:32.843329 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c6a26-6a84-425a-97ad-581640139810 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:32.844536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c6a26-6a84-425a-97ad-581640139810 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:32.844536 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c6a26-6a84-425a-97ad-581640139810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:32.900214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-971c6a26-6a84-425a-97ad-581640139810 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:32.900214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-971c6a26-6a84-425a-97ad-581640139810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:33.036109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1356/2658] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:33.049393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96642f2f-756f-4321-bd26-ba1d40aae367 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:33.049989 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96642f2f-756f-4321-bd26-ba1d40aae367 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:33.050530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96642f2f-756f-4321-bd26-ba1d40aae367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:33.223381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-96642f2f-756f-4321-bd26-ba1d40aae367 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:33.223381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-96642f2f-756f-4321-bd26-ba1d40aae367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:33.223381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1303/2659] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:33.228910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c6a26-6a84-425a-97ad-581640139810 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:33.246481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c6a26-6a84-425a-97ad-581640139810 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:33.253923 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f860a-3c3f-4ae1-810b-48dce072de2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:33.254553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f860a-3c3f-4ae1-810b-48dce072de2e admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:33.254847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f860a-3c3f-4ae1-810b-48dce072de2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:33.255109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f860a-3c3f-4ae1-810b-48dce072de2e admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:33.255771 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f860a-3c3f-4ae1-810b-48dce072de2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:12:33.256069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f860a-3c3f-4ae1-810b-48dce072de2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:33.258316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f860a-3c3f-4ae1-810b-48dce072de2e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:33.270825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1357/2660] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:12:33.277682 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96642f2f-756f-4321-bd26-ba1d40aae367 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:33.289655 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96642f2f-756f-4321-bd26-ba1d40aae367 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:33.294646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adf463-a8c5-485a-94e5-ba3627d36f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:33.295289 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adf463-a8c5-485a-94e5-ba3627d36f82 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:33.295549 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adf463-a8c5-485a-94e5-ba3627d36f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:33.295814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adf463-a8c5-485a-94e5-ba3627d36f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:33.296726 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adf463-a8c5-485a-94e5-ba3627d36f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:33.298768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adf463-a8c5-485a-94e5-ba3627d36f82 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:33.480268 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1304/2661] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:33 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:33.484415 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f860a-3c3f-4ae1-810b-48dce072de2e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:33.495593 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f860a-3c3f-4ae1-810b-48dce072de2e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:33.520792 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:33.570365 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07095e8-4e17-4317-8a8b-92b233e52d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:33.570987 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07095e8-4e17-4317-8a8b-92b233e52d69 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:33.571274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07095e8-4e17-4317-8a8b-92b233e52d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:33.571699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07095e8-4e17-4317-8a8b-92b233e52d69 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:33.572518 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07095e8-4e17-4317-8a8b-92b233e52d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:33.574466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07095e8-4e17-4317-8a8b-92b233e52d69 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:33.765911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1358/2662] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:33 2022] POST /identity/v3/users => generated 292 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:33.770167 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adf463-a8c5-485a-94e5-ba3627d36f82 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:33.785271 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adf463-a8c5-485a-94e5-ba3627d36f82 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:33.790626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb2c8e-f5ff-48b9-8f30-74c653a38af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:33.791412 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb2c8e-f5ff-48b9-8f30-74c653a38af9 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:33.791694 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb2c8e-f5ff-48b9-8f30-74c653a38af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:33.791967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb2c8e-f5ff-48b9-8f30-74c653a38af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:33.792685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb2c8e-f5ff-48b9-8f30-74c653a38af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:33.794432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb2c8e-f5ff-48b9-8f30-74c653a38af9 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:33.819257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1305/2663] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:33.821161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07095e8-4e17-4317-8a8b-92b233e52d69 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:33.833523 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07095e8-4e17-4317-8a8b-92b233e52d69 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:33.839267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cc6a1-530f-4995-9525-764e53f995d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:33.846935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cc6a1-530f-4995-9525-764e53f995d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:33.846935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cc6a1-530f-4995-9525-764e53f995d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:33.846935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cc6a1-530f-4995-9525-764e53f995d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:33.846935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cc6a1-530f-4995-9525-764e53f995d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:33.846935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cc6a1-530f-4995-9525-764e53f995d5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:33.854888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1359/2664] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:33.859958 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb2c8e-f5ff-48b9-8f30-74c653a38af9 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:33.871057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb2c8e-f5ff-48b9-8f30-74c653a38af9 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:33.883974 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddc28f-cb15-4afb-9c6c-9fd28c6eb117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:33.884744 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddc28f-cb15-4afb-9c6c-9fd28c6eb117 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:33.885064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddc28f-cb15-4afb-9c6c-9fd28c6eb117 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:33.885329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddc28f-cb15-4afb-9c6c-9fd28c6eb117 admin admin] PATH_INFO: `/v3/projects/fec29b638fc74ebfa662b6e1b22b6139/users/b4ca69e8e0c34094ba82d0a0627dbdab/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:33.918735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddc28f-cb15-4afb-9c6c-9fd28c6eb117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec29b638fc74ebfa662b6e1b22b6139, user_id=b4ca69e8e0c34094ba82d0a0627dbdab, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:33.924506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddc28f-cb15-4afb-9c6c-9fd28c6eb117 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:33.994520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:34.009466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1306/2665] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:33 2022] PUT /identity/v3/projects/fec29b638fc74ebfa662b6e1b22b6139/users/b4ca69e8e0c34094ba82d0a0627dbdab/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:34.024506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc27249-218c-4194-ab17-42c914870c0a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:34.024878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc27249-218c-4194-ab17-42c914870c0a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:34.025470 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc27249-218c-4194-ab17-42c914870c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:34.082618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2cc27249-218c-4194-ab17-42c914870c0a None None] MFA Rules not processed for user `b4ca69e8e0c34094ba82d0a0627dbdab`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:34.084559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2cc27249-218c-4194-ab17-42c914870c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:34.213963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1360/2666] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:34.232387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddc28f-cb15-4afb-9c6c-9fd28c6eb117 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:34.235312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d6ddc28f-cb15-4afb-9c6c-9fd28c6eb117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:34.316025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddc28f-cb15-4afb-9c6c-9fd28c6eb117 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:34.320545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14036f4-ae13-4005-b395-90b80eb0a8e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:34.321269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14036f4-ae13-4005-b395-90b80eb0a8e4 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:34.321627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14036f4-ae13-4005-b395-90b80eb0a8e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:34.321975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14036f4-ae13-4005-b395-90b80eb0a8e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:34.332411 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14036f4-ae13-4005-b395-90b80eb0a8e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:34.337991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14036f4-ae13-4005-b395-90b80eb0a8e4 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:34.359958 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1307/2667] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:34.425380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc27249-218c-4194-ab17-42c914870c0a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:34.579338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc27249-218c-4194-ab17-42c914870c0a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:34.594066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:34.595203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:34.596437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:34.596915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] PATH_INFO: `/v3/users/b4ca69e8e0c34094ba82d0a0627dbdab` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:34.601937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ca69e8e0c34094ba82d0a0627dbdab)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:34.608371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:34.766284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:34.931286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ca69e8e0c34094ba82d0a0627dbdab'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:34.932464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ca69e8e0c34094ba82d0a0627dbdab'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:34.957214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ca69e8e0c34094ba82d0a0627dbdab'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:34.958237 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ca69e8e0c34094ba82d0a0627dbdab'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:35.065799 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ca69e8e0c34094ba82d0a0627dbdab'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:35.127596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1361/2668] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:34 2022] DELETE /identity/v3/users/b4ca69e8e0c34094ba82d0a0627dbdab => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:35.133851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956a68d-b242-47c6-b632-52e4a3ff13d9 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:35.134315 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956a68d-b242-47c6-b632-52e4a3ff13d9 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:35.134693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956a68d-b242-47c6-b632-52e4a3ff13d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:35.169500 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e956a68d-b242-47c6-b632-52e4a3ff13d9 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:35.170831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e956a68d-b242-47c6-b632-52e4a3ff13d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:35.308029 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1308/2669] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:35.324788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:35.330021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:35.523163 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e02102-869b-4577-9ba0-d75a1a1e8254 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:35.553746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345a9d-788c-4dee-89d5-0daa3d084847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:35.555746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345a9d-788c-4dee-89d5-0daa3d084847 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:35.557100 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345a9d-788c-4dee-89d5-0daa3d084847 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:35.557785 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345a9d-788c-4dee-89d5-0daa3d084847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:35.587859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70345a9d-788c-4dee-89d5-0daa3d084847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:35.592069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70345a9d-788c-4dee-89d5-0daa3d084847 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:35.629782 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1362/2670] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:36.491062 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956a68d-b242-47c6-b632-52e4a3ff13d9 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:36.494457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e956a68d-b242-47c6-b632-52e4a3ff13d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:36.508388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956a68d-b242-47c6-b632-52e4a3ff13d9 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:36.513772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:36.514768 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:36.515266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:36.515642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] PATH_INFO: `/v3/projects/fec29b638fc74ebfa662b6e1b22b6139` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:36.517174 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec29b638fc74ebfa662b6e1b22b6139)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:36.519732 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:36.574319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345a9d-788c-4dee-89d5-0daa3d084847 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:36.576889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-70345a9d-788c-4dee-89d5-0daa3d084847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:36.599128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345a9d-788c-4dee-89d5-0daa3d084847 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:36.606592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:36.607398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:36.607833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:36.608242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] PATH_INFO: `/v3/users/ce7c4186359b46078a75c4dc260ca50d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:36.610722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7c4186359b46078a75c4dc260ca50d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:36.613689 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:36.684488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec29b638fc74ebfa662b6e1b22b6139', 'request_id': 'req-ef95abed-3ed7-47d0-b7de-bef3fba5841c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:36.696764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec29b638fc74ebfa662b6e1b22b6139', 'request_id': 'req-ef95abed-3ed7-47d0-b7de-bef3fba5841c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:36.710814 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec29b638fc74ebfa662b6e1b22b6139', 'request_id': 'req-ef95abed-3ed7-47d0-b7de-bef3fba5841c'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:36.876408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] The token cache is being invalidate because project fec29b638fc74ebfa662b6e1b22b6139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:36.880281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:36.884817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-ef95abed-3ed7-47d0-b7de-bef3fba5841c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:36.900293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1309/2671] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:36 2022] DELETE /identity/v3/projects/fec29b638fc74ebfa662b6e1b22b6139 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:36.904865 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b98ad-83c2-484a-9d69-53c690eefb2a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:36.905329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b98ad-83c2-484a-9d69-53c690eefb2a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:36.907426 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b98ad-83c2-484a-9d69-53c690eefb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:36.949981 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7c4186359b46078a75c4dc260ca50d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:36.951741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7c4186359b46078a75c4dc260ca50d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:36.955080 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-ae1b98ad-83c2-484a-9d69-53c690eefb2a None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:36.958570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ae1b98ad-83c2-484a-9d69-53c690eefb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:36.968209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7c4186359b46078a75c4dc260ca50d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:36.971497 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7c4186359b46078a75c4dc260ca50d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:36.998836 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:37.106396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1310/2672] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:37.129629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-67eea740-a3b5-45c4-bd4a-0d91d8a9c423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7c4186359b46078a75c4dc260ca50d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:37.185979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07dc21-a503-4b47-935b-e227aac2dc82 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:37.187231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07dc21-a503-4b47-935b-e227aac2dc82 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:37.187951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07dc21-a503-4b47-935b-e227aac2dc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:37.208767 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1363/2673] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:36 2022] DELETE /identity/v3/users/ce7c4186359b46078a75c4dc260ca50d => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:37.212579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f910e9-d268-4e13-9a4d-71af8ebe733f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:37.212881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f910e9-d268-4e13-9a4d-71af8ebe733f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:37.213173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f910e9-d268-4e13-9a4d-71af8ebe733f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:37.254988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-88f910e9-d268-4e13-9a4d-71af8ebe733f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:37.256609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-88f910e9-d268-4e13-9a4d-71af8ebe733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:37.266831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-8f07dc21-a503-4b47-935b-e227aac2dc82 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:37.268310 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8f07dc21-a503-4b47-935b-e227aac2dc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:37.276030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:37.369558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1364/2674] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:37.397406 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f910e9-d268-4e13-9a4d-71af8ebe733f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:37.403220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-88f910e9-d268-4e13-9a4d-71af8ebe733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:37.456126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1311/2675] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:37.464356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07dc21-a503-4b47-935b-e227aac2dc82 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:37.482777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07dc21-a503-4b47-935b-e227aac2dc82 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:37.520273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61613e8c-33a5-4bba-8a00-74d65705c1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:37.520273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61613e8c-33a5-4bba-8a00-74d65705c1db admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:37.520273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61613e8c-33a5-4bba-8a00-74d65705c1db admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:37.520273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61613e8c-33a5-4bba-8a00-74d65705c1db admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:37.521311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61613e8c-33a5-4bba-8a00-74d65705c1db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:12:37.521311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61613e8c-33a5-4bba-8a00-74d65705c1db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:37.521311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61613e8c-33a5-4bba-8a00-74d65705c1db admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:37.522254 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1312/2676] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:12:37.529634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61613e8c-33a5-4bba-8a00-74d65705c1db admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:37.543147 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f910e9-d268-4e13-9a4d-71af8ebe733f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:37.547850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61613e8c-33a5-4bba-8a00-74d65705c1db admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:37.553232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00850d5e-d97e-4314-abcf-b536255b106f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:37.554339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850d5e-d97e-4314-abcf-b536255b106f service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:37.554881 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850d5e-d97e-4314-abcf-b536255b106f service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:37.556454 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850d5e-d97e-4314-abcf-b536255b106f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:37.563376 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb74f9-170d-4441-b2b0-387b556bceb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:37.567546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb74f9-170d-4441-b2b0-387b556bceb7 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:37.567841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb74f9-170d-4441-b2b0-387b556bceb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:37.568001 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb74f9-170d-4441-b2b0-387b556bceb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:37.571941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb74f9-170d-4441-b2b0-387b556bceb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:37.575275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00850d5e-d97e-4314-abcf-b536255b106f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:37.576066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb74f9-170d-4441-b2b0-387b556bceb7 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:37.576789 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00850d5e-d97e-4314-abcf-b536255b106f service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:37.589251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1365/2677] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:37.843214 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1313/2678] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:37 2022] POST /identity/v3/projects => generated 385 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:37.849833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00850d5e-d97e-4314-abcf-b536255b106f service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:37.859086 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00850d5e-d97e-4314-abcf-b536255b106f service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:37.943662 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376734ed-6505-4344-9b4b-3d46fe52c1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:37.944731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376734ed-6505-4344-9b4b-3d46fe52c1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:37.952064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376734ed-6505-4344-9b4b-3d46fe52c1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:37.952064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376734ed-6505-4344-9b4b-3d46fe52c1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:37.952064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376734ed-6505-4344-9b4b-3d46fe52c1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:37.952064 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376734ed-6505-4344-9b4b-3d46fe52c1b5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:38.092930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1366/2679] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:37 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:38.098506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb74f9-170d-4441-b2b0-387b556bceb7 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:38.130851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb74f9-170d-4441-b2b0-387b556bceb7 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:38.148970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4d950-88ff-4cd4-974c-cb26e0227d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:38.149636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4d950-88ff-4cd4-974c-cb26e0227d5e admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:38.149944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4d950-88ff-4cd4-974c-cb26e0227d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:38.150225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4d950-88ff-4cd4-974c-cb26e0227d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:38.153970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4d950-88ff-4cd4-974c-cb26e0227d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:38.159733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4d950-88ff-4cd4-974c-cb26e0227d5e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:38.169485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1314/2680] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:38.175807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376734ed-6505-4344-9b4b-3d46fe52c1b5 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:38.188884 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376734ed-6505-4344-9b4b-3d46fe52c1b5 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:38.195756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37a131-e22c-47f8-86a1-c022c6a5b1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:38.198201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37a131-e22c-47f8-86a1-c022c6a5b1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:38.198544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37a131-e22c-47f8-86a1-c022c6a5b1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:38.198863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37a131-e22c-47f8-86a1-c022c6a5b1a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:38.199629 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37a131-e22c-47f8-86a1-c022c6a5b1a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:38.201403 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37a131-e22c-47f8-86a1-c022c6a5b1a0 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:38.217306 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1367/2681] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:38.217756 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4d950-88ff-4cd4-974c-cb26e0227d5e admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:38.229232 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4d950-88ff-4cd4-974c-cb26e0227d5e admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:38.242683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52741853-da7e-4636-b28d-becd5838b77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:38.243590 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52741853-da7e-4636-b28d-becd5838b77d admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:38.248598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52741853-da7e-4636-b28d-becd5838b77d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:38.249564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37a131-e22c-47f8-86a1-c022c6a5b1a0 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:38.250056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52741853-da7e-4636-b28d-becd5838b77d admin admin] PATH_INFO: `/v3/projects/82f69fe7b30042b1a17aa3de8e613188/users/415973a355744b78a376ff5e2bbda1e8/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:38.252344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ed37a131-e22c-47f8-86a1-c022c6a5b1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:38.281297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37a131-e22c-47f8-86a1-c022c6a5b1a0 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:38.292873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52741853-da7e-4636-b28d-becd5838b77d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f69fe7b30042b1a17aa3de8e613188, user_id=415973a355744b78a376ff5e2bbda1e8, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:38.300341 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52741853-da7e-4636-b28d-becd5838b77d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:38.311432 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:38.313203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:38.313673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:38.314060 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] PATH_INFO: `/v3/projects/6e65cf18a7da47f4a61c47ee2dccea8e` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:38.315665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e65cf18a7da47f4a61c47ee2dccea8e)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:38.318336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:38.404970 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1315/2682] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:38 2022] PUT /identity/v3/projects/82f69fe7b30042b1a17aa3de8e613188/users/415973a355744b78a376ff5e2bbda1e8/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:38.416491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2859e-faa5-48ee-978b-218f9ce68bb6 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:38.417153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2859e-faa5-48ee-978b-218f9ce68bb6 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:38.417704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2859e-faa5-48ee-978b-218f9ce68bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:38.463134 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b0a2859e-faa5-48ee-978b-218f9ce68bb6 None None] MFA Rules not processed for user `415973a355744b78a376ff5e2bbda1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:38.464896 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b0a2859e-faa5-48ee-978b-218f9ce68bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:38.531270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:38.533997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:38.575957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1316/2683] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:38.576530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e65cf18a7da47f4a61c47ee2dccea8e', 'request_id': 'req-cf19e449-cd22-4924-a761-fa9ebda93864'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:38.581191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2859e-faa5-48ee-978b-218f9ce68bb6 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:38.590485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e65cf18a7da47f4a61c47ee2dccea8e', 'request_id': 'req-cf19e449-cd22-4924-a761-fa9ebda93864'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:38.687756 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e65cf18a7da47f4a61c47ee2dccea8e', 'request_id': 'req-cf19e449-cd22-4924-a761-fa9ebda93864'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:38.689078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2859e-faa5-48ee-978b-218f9ce68bb6 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:38.731548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] The token cache is being invalidate because project 6e65cf18a7da47f4a61c47ee2dccea8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:38.732835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:38.734982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:38.738878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1368/2684] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:38 2022] DELETE /identity/v3/projects/6e65cf18a7da47f4a61c47ee2dccea8e => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:38.746372 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:38.749805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:38.829641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd827ff-9771-4ccc-9678-9408c73fa9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:38.829641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd827ff-9771-4ccc-9678-9408c73fa9ad admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:38.829641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd827ff-9771-4ccc-9678-9408c73fa9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:38.829641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd827ff-9771-4ccc-9678-9408c73fa9ad admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:38.830292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd827ff-9771-4ccc-9678-9408c73fa9ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:38.830292 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd827ff-9771-4ccc-9678-9408c73fa9ad admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:38.878505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19e449-cd22-4924-a761-fa9ebda93864 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:38.884161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:38.884845 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:38.885124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:38.885391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] PATH_INFO: `/v3/users/ad9d12a802fe4cb3bc9113cf0cb72e5e` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:38.886583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad9d12a802fe4cb3bc9113cf0cb72e5e)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:38.888610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:38.970898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1317/2685] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:38 2022] POST /identity/v3/projects => generated 381 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:38.975245 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd827ff-9771-4ccc-9678-9408c73fa9ad admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:38.977536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0cd827ff-9771-4ccc-9678-9408c73fa9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:39.056321 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd827ff-9771-4ccc-9678-9408c73fa9ad admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:39.061452 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18216bbd-2abb-481c-ac8c-feb1eb440125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:39.062069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18216bbd-2abb-481c-ac8c-feb1eb440125 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:39.062632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18216bbd-2abb-481c-ac8c-feb1eb440125 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:39.062914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18216bbd-2abb-481c-ac8c-feb1eb440125 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:39.063559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18216bbd-2abb-481c-ac8c-feb1eb440125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:39.065582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18216bbd-2abb-481c-ac8c-feb1eb440125 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:39.152183 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad9d12a802fe4cb3bc9113cf0cb72e5e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:39.153139 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad9d12a802fe4cb3bc9113cf0cb72e5e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:39.161240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad9d12a802fe4cb3bc9113cf0cb72e5e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:39.162017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad9d12a802fe4cb3bc9113cf0cb72e5e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:39.204546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1318/2686] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:38 2022] POST /identity/v3/users => generated 291 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:39.204546 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18216bbd-2abb-481c-ac8c-feb1eb440125 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:39.206857 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-18216bbd-2abb-481c-ac8c-feb1eb440125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:39.221186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18216bbd-2abb-481c-ac8c-feb1eb440125 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:39.229977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd42ea-7347-4404-8e67-29e2f07c6ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:39.231533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd42ea-7347-4404-8e67-29e2f07c6ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:39.232070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd42ea-7347-4404-8e67-29e2f07c6ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:39.232613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd42ea-7347-4404-8e67-29e2f07c6ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:39.233747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd42ea-7347-4404-8e67-29e2f07c6ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:39.237005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd42ea-7347-4404-8e67-29e2f07c6ed1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:39.239348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:39.255405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1319/2687] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:39.259797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd42ea-7347-4404-8e67-29e2f07c6ed1 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:39.271491 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd42ea-7347-4404-8e67-29e2f07c6ed1 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:39.276486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be0e8f-f0ff-46b0-9438-d9c0f9e231d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:39.277132 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be0e8f-f0ff-46b0-9438-d9c0f9e231d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:39.277420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be0e8f-f0ff-46b0-9438-d9c0f9e231d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:39.277679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be0e8f-f0ff-46b0-9438-d9c0f9e231d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:39.278326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be0e8f-f0ff-46b0-9438-d9c0f9e231d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:39.280728 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be0e8f-f0ff-46b0-9438-d9c0f9e231d8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:39.292613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1320/2688] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:39.296862 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be0e8f-f0ff-46b0-9438-d9c0f9e231d8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:39.307505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be0e8f-f0ff-46b0-9438-d9c0f9e231d8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:39.313060 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2869c0-fb63-4fb0-95d7-49dac937b345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:39.313737 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2869c0-fb63-4fb0-95d7-49dac937b345 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:39.314022 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2869c0-fb63-4fb0-95d7-49dac937b345 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:39.314291 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2869c0-fb63-4fb0-95d7-49dac937b345 admin admin] PATH_INFO: `/v3/projects/8627cf2f447740ddaf86afe796929be0/users/f3515b3708a342fdb8dafe8cb7b6d1b1/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:39.330618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2869c0-fb63-4fb0-95d7-49dac937b345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8627cf2f447740ddaf86afe796929be0, user_id=f3515b3708a342fdb8dafe8cb7b6d1b1, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:39.333386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2869c0-fb63-4fb0-95d7-49dac937b345 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:39.386755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-8b5f3c85-e9d3-4a48-801b-3e5013f7b0e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad9d12a802fe4cb3bc9113cf0cb72e5e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:39.427387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1369/2689] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:38 2022] DELETE /identity/v3/users/ad9d12a802fe4cb3bc9113cf0cb72e5e => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:39.496011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1321/2690] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:39 2022] PUT /identity/v3/projects/8627cf2f447740ddaf86afe796929be0/users/f3515b3708a342fdb8dafe8cb7b6d1b1/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:39.517630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa0c65-8ec2-4ce6-ad3f-b704cbe04459 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:39.518183 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa0c65-8ec2-4ce6-ad3f-b704cbe04459 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:39.518873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa0c65-8ec2-4ce6-ad3f-b704cbe04459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:39.573587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-d8fa0c65-8ec2-4ce6-ad3f-b704cbe04459 None None] MFA Rules not processed for user `f3515b3708a342fdb8dafe8cb7b6d1b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:39.575029 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-d8fa0c65-8ec2-4ce6-ad3f-b704cbe04459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:39.638894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:39.659267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1370/2691] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:39.664745 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2869c0-fb63-4fb0-95d7-49dac937b345 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:39.748641 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2869c0-fb63-4fb0-95d7-49dac937b345 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:39.755488 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20568430-d457-49c3-9dde-546386faa249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:39.760557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20568430-d457-49c3-9dde-546386faa249 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:39.760557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20568430-d457-49c3-9dde-546386faa249 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:39.760557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20568430-d457-49c3-9dde-546386faa249 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:39.760557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20568430-d457-49c3-9dde-546386faa249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:39.763312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20568430-d457-49c3-9dde-546386faa249 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:39.950913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1322/2692] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:39 2022] POST /identity/v3/projects => generated 383 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:39.955774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa0c65-8ec2-4ce6-ad3f-b704cbe04459 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:39.970280 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa0c65-8ec2-4ce6-ad3f-b704cbe04459 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:40.044322 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301da9c-b96c-4085-8433-24027f5191ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:40.045419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301da9c-b96c-4085-8433-24027f5191ab admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:40.045419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301da9c-b96c-4085-8433-24027f5191ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:40.045946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301da9c-b96c-4085-8433-24027f5191ab admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:40.046555 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301da9c-b96c-4085-8433-24027f5191ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:40.048667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301da9c-b96c-4085-8433-24027f5191ab admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:40.170843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1371/2693] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:39 2022] POST /identity/v3/users => generated 298 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:40.175567 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20568430-d457-49c3-9dde-546386faa249 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:40.188227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20568430-d457-49c3-9dde-546386faa249 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:40.193344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b2d30-b3f0-4980-bdce-4d4ce0770c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:40.193980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b2d30-b3f0-4980-bdce-4d4ce0770c53 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:40.194261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b2d30-b3f0-4980-bdce-4d4ce0770c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:40.194537 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b2d30-b3f0-4980-bdce-4d4ce0770c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:40.195198 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b2d30-b3f0-4980-bdce-4d4ce0770c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:40.196991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b2d30-b3f0-4980-bdce-4d4ce0770c53 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:40.209219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1323/2694] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:40.214235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301da9c-b96c-4085-8433-24027f5191ab admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:40.225381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301da9c-b96c-4085-8433-24027f5191ab admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:40.231559 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8e277-9102-4ecd-8357-418e2196f489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:40.232351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8e277-9102-4ecd-8357-418e2196f489 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:40.232641 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8e277-9102-4ecd-8357-418e2196f489 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:40.232926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8e277-9102-4ecd-8357-418e2196f489 admin admin] PATH_INFO: `/v3/projects/4e2e2a23f0b04304842ed410fa48ac1f/users/472f15d601da4469b56ee23ba42e1d53/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:40.254257 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8e277-9102-4ecd-8357-418e2196f489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2e2a23f0b04304842ed410fa48ac1f, user_id=472f15d601da4469b56ee23ba42e1d53, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:40.257240 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8e277-9102-4ecd-8357-418e2196f489 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:40.364746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1372/2695] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:40 2022] PUT /identity/v3/projects/4e2e2a23f0b04304842ed410fa48ac1f/users/472f15d601da4469b56ee23ba42e1d53/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:40.369681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b2d30-b3f0-4980-bdce-4d4ce0770c53 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:40.380710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8e277-9102-4ecd-8357-418e2196f489 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:40.382766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-14e8e277-9102-4ecd-8357-418e2196f489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:40.387225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b2d30-b3f0-4980-bdce-4d4ce0770c53 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:40.489006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8e277-9102-4ecd-8357-418e2196f489 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:40.490311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c137cd-3ee9-43d7-b877-53160a2e9152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:40.490906 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c137cd-3ee9-43d7-b877-53160a2e9152 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:40.491190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c137cd-3ee9-43d7-b877-53160a2e9152 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:40.492128 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c137cd-3ee9-43d7-b877-53160a2e9152 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:40.492824 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c137cd-3ee9-43d7-b877-53160a2e9152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:40.494719 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c137cd-3ee9-43d7-b877-53160a2e9152 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:40.495938 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:40.496661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:40.496949 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:40.497264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] PATH_INFO: `/v3/projects/d1a3e973af424e2d9fdd6dea88f8ac97` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:40.498505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a3e973af424e2d9fdd6dea88f8ac97)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:40.501001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:40.507359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1324/2696] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:40.511835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c137cd-3ee9-43d7-b877-53160a2e9152 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:40.523360 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c137cd-3ee9-43d7-b877-53160a2e9152 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:40.528993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11add979-7d9a-4851-9f88-9f78738531d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:40.529887 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add979-7d9a-4851-9f88-9f78738531d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:40.530262 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add979-7d9a-4851-9f88-9f78738531d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:40.530637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add979-7d9a-4851-9f88-9f78738531d4 admin admin] PATH_INFO: `/v3/projects/4e2e2a23f0b04304842ed410fa48ac1f/users/472f15d601da4469b56ee23ba42e1d53/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:40.533890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11add979-7d9a-4851-9f88-9f78738531d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2e2a23f0b04304842ed410fa48ac1f, user_id=472f15d601da4469b56ee23ba42e1d53, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:40.539099 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11add979-7d9a-4851-9f88-9f78738531d4 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:40.556631 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1325/2697] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:40 2022] PUT /identity/v3/projects/4e2e2a23f0b04304842ed410fa48ac1f/users/472f15d601da4469b56ee23ba42e1d53/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:40.575342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0a5b7-2480-4bd4-80e0-e0dc1b898057 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:40.575892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0a5b7-2480-4bd4-80e0-e0dc1b898057 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:40.576381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0a5b7-2480-4bd4-80e0-e0dc1b898057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:40.614194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-3bb0a5b7-2480-4bd4-80e0-e0dc1b898057 None None] MFA Rules not processed for user `472f15d601da4469b56ee23ba42e1d53`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:40.615504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3bb0a5b7-2480-4bd4-80e0-e0dc1b898057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:40.622570 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:40.674646 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:40.690267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a3e973af424e2d9fdd6dea88f8ac97', 'request_id': 'req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:40.700936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a3e973af424e2d9fdd6dea88f8ac97', 'request_id': 'req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:40.727994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1326/2698] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:40.745612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0a5b7-2480-4bd4-80e0-e0dc1b898057 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:40.748231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3bb0a5b7-2480-4bd4-80e0-e0dc1b898057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:40.812116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a3e973af424e2d9fdd6dea88f8ac97', 'request_id': 'req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:40.836054 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] The token cache is being invalidate because project d1a3e973af424e2d9fdd6dea88f8ac97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:40.836720 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:40.838031 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-1a0cbc92-bebe-4dab-92af-a3a7dd705d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:40.844197 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1373/2699] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:40 2022] DELETE /identity/v3/projects/d1a3e973af424e2d9fdd6dea88f8ac97 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:40.853674 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a278c-dcb2-45dd-99ba-b4fd513c5558 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:40.855327 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a278c-dcb2-45dd-99ba-b4fd513c5558 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:40.855866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a278c-dcb2-45dd-99ba-b4fd513c5558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:40.862180 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0a5b7-2480-4bd4-80e0-e0dc1b898057 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:40.882006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-526a278c-dcb2-45dd-99ba-b4fd513c5558 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:40.882644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-526a278c-dcb2-45dd-99ba-b4fd513c5558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:40.975386 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed4e08-c357-4b5d-a0b9-0c42edca2128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:40.976536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed4e08-c357-4b5d-a0b9-0c42edca2128 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:40.976901 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed4e08-c357-4b5d-a0b9-0c42edca2128 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:40.977673 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed4e08-c357-4b5d-a0b9-0c42edca2128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:40.979723 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-35ed4e08-c357-4b5d-a0b9-0c42edca2128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:41.004545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1374/2700] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:41.006128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a278c-dcb2-45dd-99ba-b4fd513c5558 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:41.008364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-526a278c-dcb2-45dd-99ba-b4fd513c5558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:41.092749 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed4e08-c357-4b5d-a0b9-0c42edca2128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:41.094270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed4e08-c357-4b5d-a0b9-0c42edca2128 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:41.105149 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1327/2701] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:41.106764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a39396-b51d-4909-a984-28faa1980377 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:41.107055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a39396-b51d-4909-a984-28faa1980377 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:41.107337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a39396-b51d-4909-a984-28faa1980377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:41.134362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-74a39396-b51d-4909-a984-28faa1980377 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:41.134362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-74a39396-b51d-4909-a984-28faa1980377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:41.142364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a278c-dcb2-45dd-99ba-b4fd513c5558 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:41.146490 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1328/2702] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:41.149373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a39396-b51d-4909-a984-28faa1980377 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:41.151228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b66a7-1603-4a0b-b8e4-b1e732d09838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:41.151859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b66a7-1603-4a0b-b8e4-b1e732d09838 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:41.152134 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b66a7-1603-4a0b-b8e4-b1e732d09838 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:41.152430 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b66a7-1603-4a0b-b8e4-b1e732d09838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:41.154450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-bf6b66a7-1603-4a0b-b8e4-b1e732d09838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:41.161965 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a39396-b51d-4909-a984-28faa1980377 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:41.171136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9286fa5-36c8-44c3-a0e5-62e1d40fa768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:41.172236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9286fa5-36c8-44c3-a0e5-62e1d40fa768 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:41.173790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9286fa5-36c8-44c3-a0e5-62e1d40fa768 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:41.174424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9286fa5-36c8-44c3-a0e5-62e1d40fa768 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:41.175903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9286fa5-36c8-44c3-a0e5-62e1d40fa768 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:12:41.176515 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9286fa5-36c8-44c3-a0e5-62e1d40fa768 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:41.186057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9286fa5-36c8-44c3-a0e5-62e1d40fa768 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:41.186057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=100 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:41.203999 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1329/2703] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:12:41.209760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9286fa5-36c8-44c3-a0e5-62e1d40fa768 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:41.223995 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9286fa5-36c8-44c3-a0e5-62e1d40fa768 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:41.230389 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25949de-41ab-4e49-bc5a-d1780c8ac105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:41.231617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25949de-41ab-4e49-bc5a-d1780c8ac105 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:41.231617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25949de-41ab-4e49-bc5a-d1780c8ac105 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:41.231850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25949de-41ab-4e49-bc5a-d1780c8ac105 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:41.232363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25949de-41ab-4e49-bc5a-d1780c8ac105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:41.234508 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25949de-41ab-4e49-bc5a-d1780c8ac105 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:41.280624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b66a7-1603-4a0b-b8e4-b1e732d09838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:41.282184 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b66a7-1603-4a0b-b8e4-b1e732d09838 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:41.295063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1375/2704] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:41.384269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1330/2705] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:41 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:41.388275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b66a7-1603-4a0b-b8e4-b1e732d09838 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:41.399857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b66a7-1603-4a0b-b8e4-b1e732d09838 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:41.421225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:41.474142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a48eaf-6277-45d2-bc86-ba162e5e20c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:41.474142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a48eaf-6277-45d2-bc86-ba162e5e20c1 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:41.474142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a48eaf-6277-45d2-bc86-ba162e5e20c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:41.474142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a48eaf-6277-45d2-bc86-ba162e5e20c1 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:41.474693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a48eaf-6277-45d2-bc86-ba162e5e20c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:41.474693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a48eaf-6277-45d2-bc86-ba162e5e20c1 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:41.650919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1376/2706] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:41 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:41.652307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25949de-41ab-4e49-bc5a-d1780c8ac105 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:41.668425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25949de-41ab-4e49-bc5a-d1780c8ac105 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:41.673598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957ec0a-72f8-4c07-bd26-983f8dc100c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:41.674222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957ec0a-72f8-4c07-bd26-983f8dc100c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:41.674504 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957ec0a-72f8-4c07-bd26-983f8dc100c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:41.674777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957ec0a-72f8-4c07-bd26-983f8dc100c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:41.675457 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957ec0a-72f8-4c07-bd26-983f8dc100c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:41.677335 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957ec0a-72f8-4c07-bd26-983f8dc100c7 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:41.697501 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1331/2707] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:41.700319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a48eaf-6277-45d2-bc86-ba162e5e20c1 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:41.711862 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a48eaf-6277-45d2-bc86-ba162e5e20c1 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:41.717391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5d238-cfc5-42fd-b316-c8aae3529472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:41.718014 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5d238-cfc5-42fd-b316-c8aae3529472 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:41.718296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5d238-cfc5-42fd-b316-c8aae3529472 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:41.718570 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5d238-cfc5-42fd-b316-c8aae3529472 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:41.719242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5d238-cfc5-42fd-b316-c8aae3529472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:41.720976 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede5d238-cfc5-42fd-b316-c8aae3529472 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:41.738741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1377/2708] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:41.746131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957ec0a-72f8-4c07-bd26-983f8dc100c7 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:41.757658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957ec0a-72f8-4c07-bd26-983f8dc100c7 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:41.762834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9c316-b9b3-4f4e-b2c8-8dc4de92bcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:41.763525 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9c316-b9b3-4f4e-b2c8-8dc4de92bcd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:41.763808 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9c316-b9b3-4f4e-b2c8-8dc4de92bcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:41.764085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9c316-b9b3-4f4e-b2c8-8dc4de92bcd5 admin admin] PATH_INFO: `/v3/projects/b79f7e7bc370415aad89c8101721064b/users/cb65a8f3f1e144ad8fdfc75c37f706eb/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:41.786158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9c316-b9b3-4f4e-b2c8-8dc4de92bcd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b79f7e7bc370415aad89c8101721064b, user_id=cb65a8f3f1e144ad8fdfc75c37f706eb, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:41.790751 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9c316-b9b3-4f4e-b2c8-8dc4de92bcd5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:41.963634 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1332/2709] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:41 2022] PUT /identity/v3/projects/b79f7e7bc370415aad89c8101721064b/users/cb65a8f3f1e144ad8fdfc75c37f706eb/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:41.976189 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c3064-19f9-43ea-b18a-e9a921ba9fab None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:41.976560 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c3064-19f9-43ea-b18a-e9a921ba9fab None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:41.976753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c3064-19f9-43ea-b18a-e9a921ba9fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:42.009853 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1d7c3064-19f9-43ea-b18a-e9a921ba9fab None None] MFA Rules not processed for user `cb65a8f3f1e144ad8fdfc75c37f706eb`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:42.011094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1d7c3064-19f9-43ea-b18a-e9a921ba9fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:42.094467 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1378/2710] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:42.113124 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9c316-b9b3-4f4e-b2c8-8dc4de92bcd5 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:42.115574 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-92c9c316-b9b3-4f4e-b2c8-8dc4de92bcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:42.230688 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9c316-b9b3-4f4e-b2c8-8dc4de92bcd5 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:42.237224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258af12-2753-4d65-89b1-c8776808d3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:42.238145 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258af12-2753-4d65-89b1-c8776808d3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:42.238781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258af12-2753-4d65-89b1-c8776808d3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:42.240528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258af12-2753-4d65-89b1-c8776808d3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:42.253252 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258af12-2753-4d65-89b1-c8776808d3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:42.255660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258af12-2753-4d65-89b1-c8776808d3c9 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:42.279632 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1333/2711] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:43.726481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c3064-19f9-43ea-b18a-e9a921ba9fab None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:43.728045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1d7c3064-19f9-43ea-b18a-e9a921ba9fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:43.741892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c3064-19f9-43ea-b18a-e9a921ba9fab None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:43.746661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef94e2-d080-4159-a52b-5dc825008ed7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:43.747300 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef94e2-d080-4159-a52b-5dc825008ed7 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:43.747580 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef94e2-d080-4159-a52b-5dc825008ed7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:43.747855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef94e2-d080-4159-a52b-5dc825008ed7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:43.756579 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3cef94e2-d080-4159-a52b-5dc825008ed7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:43.852773 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef94e2-d080-4159-a52b-5dc825008ed7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:43.854618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef94e2-d080-4159-a52b-5dc825008ed7 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:43.867676 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1379/2712] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:46.756266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258af12-2753-4d65-89b1-c8776808d3c9 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:46.766177 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258af12-2753-4d65-89b1-c8776808d3c9 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:46.770090 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72425c50-c7d5-45ec-961a-a5e51230463a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:46.770712 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72425c50-c7d5-45ec-961a-a5e51230463a service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:46.771040 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72425c50-c7d5-45ec-961a-a5e51230463a service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:46.771357 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72425c50-c7d5-45ec-961a-a5e51230463a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:46.773303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-72425c50-c7d5-45ec-961a-a5e51230463a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:46.854997 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72425c50-c7d5-45ec-961a-a5e51230463a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:46.856776 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72425c50-c7d5-45ec-961a-a5e51230463a service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:46.875818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1334/2713] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:48.110740 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa9998-08f2-4d38-9fad-62aae10cbc4e None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:48.113279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa9998-08f2-4d38-9fad-62aae10cbc4e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:48.113279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa9998-08f2-4d38-9fad-62aae10cbc4e None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:48.113873 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1380/2714] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:12:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:12:48.130110 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84462faf-78a2-41cb-88bc-4aa86573093a None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:48.130810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84462faf-78a2-41cb-88bc-4aa86573093a None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:48.130810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84462faf-78a2-41cb-88bc-4aa86573093a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:48.197074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-84462faf-78a2-41cb-88bc-4aa86573093a None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:48.199107 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-84462faf-78a2-41cb-88bc-4aa86573093a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:48.220147 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:48.293373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1335/2715] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:12:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:48.309092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa9998-08f2-4d38-9fad-62aae10cbc4e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:48.312625 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-feaa9998-08f2-4d38-9fad-62aae10cbc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:48.352533 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:48.394687 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa9998-08f2-4d38-9fad-62aae10cbc4e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:48.402655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9a929-529c-4788-83fa-2acdfd0aaf0c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454fb811f6e34e1d881d4948ede058b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:48.402655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9a929-529c-4788-83fa-2acdfd0aaf0c service glance] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:48.402655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9a929-529c-4788-83fa-2acdfd0aaf0c service glance] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:48.402655 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9a929-529c-4788-83fa-2acdfd0aaf0c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:48.414975 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9a929-529c-4788-83fa-2acdfd0aaf0c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:48.416495 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9a929-529c-4788-83fa-2acdfd0aaf0c service glance] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:48.429668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1381/2716] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:48.609444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84462faf-78a2-41cb-88bc-4aa86573093a None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:48.708414 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239da58-3200-44ee-8c52-4a0ef9dd6da2 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:48.715399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239da58-3200-44ee-8c52-4a0ef9dd6da2 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:48.715399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239da58-3200-44ee-8c52-4a0ef9dd6da2 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:48.715399 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1382/2717] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:12:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:12:48.717671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb2d40-7118-4015-8d41-19e5d380646e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:48.718178 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb2d40-7118-4015-8d41-19e5d380646e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:48.718540 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb2d40-7118-4015-8d41-19e5d380646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:48.727724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84462faf-78a2-41cb-88bc-4aa86573093a None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:48.735955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:48.737055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:48.737403 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:48.737852 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] PATH_INFO: `/v3/users/cb65a8f3f1e144ad8fdfc75c37f706eb` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:48.739402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb65a8f3f1e144ad8fdfc75c37f706eb)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:48.743338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-3beb2d40-7118-4015-8d41-19e5d380646e None None] MFA Rules not processed for user `073e43c049224af29e9f6d426ea47e84`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:48.744087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:48.744719 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3beb2d40-7118-4015-8d41-19e5d380646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:48.821755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1383/2718] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:12:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:48.837723 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb2d40-7118-4015-8d41-19e5d380646e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:48.847268 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb2d40-7118-4015-8d41-19e5d380646e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:48.851671 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40d870-2c59-4965-b60c-65630235d6a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:48.852371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d870-2c59-4965-b60c-65630235d6a2 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:48.852595 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d870-2c59-4965-b60c-65630235d6a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:48.852856 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d870-2c59-4965-b60c-65630235d6a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:48.861379 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40d870-2c59-4965-b60c-65630235d6a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:48.862834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40d870-2c59-4965-b60c-65630235d6a2 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:48.873688 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1384/2719] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:48.979885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9a515-85f8-445e-b992-bb70fdec830e None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:48.980308 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9a515-85f8-445e-b992-bb70fdec830e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:48.980794 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9a515-85f8-445e-b992-bb70fdec830e None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:48.982955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1385/2720] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:12:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:12:48.989622 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db5366-9bee-47c4-8443-870a92f8d15b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:48.990008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db5366-9bee-47c4-8443-870a92f8d15b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:48.990791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db5366-9bee-47c4-8443-870a92f8d15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:49.022621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-41db5366-9bee-47c4-8443-870a92f8d15b None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:49.023126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb65a8f3f1e144ad8fdfc75c37f706eb'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:49.024618 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-41db5366-9bee-47c4-8443-870a92f8d15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:49.032013 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb65a8f3f1e144ad8fdfc75c37f706eb'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:49.033658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb65a8f3f1e144ad8fdfc75c37f706eb'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:49.048865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1386/2721] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:12:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:49.054325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db5366-9bee-47c4-8443-870a92f8d15b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:49.064564 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db5366-9bee-47c4-8443-870a92f8d15b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:49.068667 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd59d3-49ca-4488-a65f-bdc37131a1ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:49.069330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd59d3-49ca-4488-a65f-bdc37131a1ff service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:49.069607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd59d3-49ca-4488-a65f-bdc37131a1ff service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:49.069945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd59d3-49ca-4488-a65f-bdc37131a1ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:49.071998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c4fd59d3-49ca-4488-a65f-bdc37131a1ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:49.087251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd59d3-49ca-4488-a65f-bdc37131a1ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:49.088731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd59d3-49ca-4488-a65f-bdc37131a1ff service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:49.102614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1387/2722] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:49.140359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb65a8f3f1e144ad8fdfc75c37f706eb'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:49.165410 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb65a8f3f1e144ad8fdfc75c37f706eb'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:49.174429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1336/2723] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:48 2022] DELETE /identity/v3/users/cb65a8f3f1e144ad8fdfc75c37f706eb => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:49.179193 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4b1b5-cc4f-4017-bda0-94fad24ac7a3 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:49.179712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4b1b5-cc4f-4017-bda0-94fad24ac7a3 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:49.179712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4b1b5-cc4f-4017-bda0-94fad24ac7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:49.200739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-7ef4b1b5-cc4f-4017-bda0-94fad24ac7a3 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:49.202186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7ef4b1b5-cc4f-4017-bda0-94fad24ac7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:49.298722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1388/2724] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:49.314019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:49.315555 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:49.358046 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:49.388061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dc7db-8f15-45ea-8abf-206d2d5a0ba5 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:49.392267 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cc82d-752d-471f-80dd-c23cb39866eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:49.392967 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cc82d-752d-471f-80dd-c23cb39866eb service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:49.393312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cc82d-752d-471f-80dd-c23cb39866eb service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:49.393774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cc82d-752d-471f-80dd-c23cb39866eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:49.405285 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cc82d-752d-471f-80dd-c23cb39866eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:49.406758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cc82d-752d-471f-80dd-c23cb39866eb service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:49.418466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1337/2725] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:49.922507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4b1b5-cc4f-4017-bda0-94fad24ac7a3 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:49.925170 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7ef4b1b5-cc4f-4017-bda0-94fad24ac7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:49.945046 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4b1b5-cc4f-4017-bda0-94fad24ac7a3 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:49.949973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:49.950642 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:49.950926 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:49.951223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] PATH_INFO: `/v3/projects/b79f7e7bc370415aad89c8101721064b` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:49.952506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b79f7e7bc370415aad89c8101721064b)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:49.954517 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:50.102504 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:50.113499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b79f7e7bc370415aad89c8101721064b', 'request_id': 'req-d6a0f195-6d57-4630-a00f-fd156ad2a277'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:50.123392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b79f7e7bc370415aad89c8101721064b', 'request_id': 'req-d6a0f195-6d57-4630-a00f-fd156ad2a277'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:50.425990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b79f7e7bc370415aad89c8101721064b', 'request_id': 'req-d6a0f195-6d57-4630-a00f-fd156ad2a277'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:50.441876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] The token cache is being invalidate because project b79f7e7bc370415aad89c8101721064b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:50.442186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:50.442914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-d6a0f195-6d57-4630-a00f-fd156ad2a277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:50.445047 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1389/2726] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:49 2022] DELETE /identity/v3/projects/b79f7e7bc370415aad89c8101721064b => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:50.453422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02196033-85ad-4bf8-a66f-afa033152ade None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:50.453960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02196033-85ad-4bf8-a66f-afa033152ade None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:50.454393 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02196033-85ad-4bf8-a66f-afa033152ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:50.488801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-02196033-85ad-4bf8-a66f-afa033152ade None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:50.490484 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-02196033-85ad-4bf8-a66f-afa033152ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:50.614133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1338/2727] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:50.628596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12dbfe-6a45-48e0-91cb-f6448f9f3dd9 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:50.629038 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12dbfe-6a45-48e0-91cb-f6448f9f3dd9 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:50.629450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12dbfe-6a45-48e0-91cb-f6448f9f3dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:50.668585 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-7e12dbfe-6a45-48e0-91cb-f6448f9f3dd9 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:50.670825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7e12dbfe-6a45-48e0-91cb-f6448f9f3dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:50.687455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1390/2728] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:50.693272 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02196033-85ad-4bf8-a66f-afa033152ade None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:50.712801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02196033-85ad-4bf8-a66f-afa033152ade None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:50.721693 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc13449-0dfa-47bb-94bc-d7356675f304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:50.722729 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc13449-0dfa-47bb-94bc-d7356675f304 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:50.723227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc13449-0dfa-47bb-94bc-d7356675f304 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:50.723725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc13449-0dfa-47bb-94bc-d7356675f304 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:50.725388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc13449-0dfa-47bb-94bc-d7356675f304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:12:50.725850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc13449-0dfa-47bb-94bc-d7356675f304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:50.729030 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc13449-0dfa-47bb-94bc-d7356675f304 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:50.771282 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1339/2729] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:12:50.776919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12dbfe-6a45-48e0-91cb-f6448f9f3dd9 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:50.795389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12dbfe-6a45-48e0-91cb-f6448f9f3dd9 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:50.804010 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f242c0-059b-4f21-ba63-c8f73a50edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:50.805061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f242c0-059b-4f21-ba63-c8f73a50edae admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:50.805397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f242c0-059b-4f21-ba63-c8f73a50edae admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:50.805705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f242c0-059b-4f21-ba63-c8f73a50edae admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:50.806401 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f242c0-059b-4f21-ba63-c8f73a50edae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:50.810448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f242c0-059b-4f21-ba63-c8f73a50edae admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:50.947288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1391/2730] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:50 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:50.952477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc13449-0dfa-47bb-94bc-d7356675f304 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:50.970575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc13449-0dfa-47bb-94bc-d7356675f304 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:51.045874 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620488e-ecc4-4a0f-8bb6-9469b0025996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:51.046526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620488e-ecc4-4a0f-8bb6-9469b0025996 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:51.046806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620488e-ecc4-4a0f-8bb6-9469b0025996 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:51.047079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620488e-ecc4-4a0f-8bb6-9469b0025996 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:51.047831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620488e-ecc4-4a0f-8bb6-9469b0025996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:51.050061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620488e-ecc4-4a0f-8bb6-9469b0025996 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:51.164859 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1340/2731] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:50 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:12:51.174242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f242c0-059b-4f21-ba63-c8f73a50edae admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:51.192946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f242c0-059b-4f21-ba63-c8f73a50edae admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:51.200930 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fecd13-896c-4a5b-ae93-c28c7798cc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:51.201917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fecd13-896c-4a5b-ae93-c28c7798cc48 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:51.202372 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fecd13-896c-4a5b-ae93-c28c7798cc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:51.202844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fecd13-896c-4a5b-ae93-c28c7798cc48 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:51.203985 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fecd13-896c-4a5b-ae93-c28c7798cc48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:51.206957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fecd13-896c-4a5b-ae93-c28c7798cc48 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:51.221346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1392/2732] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:51.226791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620488e-ecc4-4a0f-8bb6-9469b0025996 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:51.247061 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620488e-ecc4-4a0f-8bb6-9469b0025996 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:51.255609 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847c572-c05b-428f-8238-c4904b758e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:51.256817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847c572-c05b-428f-8238-c4904b758e75 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:51.257317 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847c572-c05b-428f-8238-c4904b758e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:51.257908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847c572-c05b-428f-8238-c4904b758e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:51.258966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847c572-c05b-428f-8238-c4904b758e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:51.261962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847c572-c05b-428f-8238-c4904b758e75 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:51.282086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1341/2733] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:51.286592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fecd13-896c-4a5b-ae93-c28c7798cc48 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:51.297991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fecd13-896c-4a5b-ae93-c28c7798cc48 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:51.302867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36467fa-8bd2-4de8-b907-65ca1052dce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:51.304146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36467fa-8bd2-4de8-b907-65ca1052dce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:51.304457 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36467fa-8bd2-4de8-b907-65ca1052dce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:51.304762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36467fa-8bd2-4de8-b907-65ca1052dce1 admin admin] PATH_INFO: `/v3/projects/23ef9812a3194dcb9c88e284be7e3982/users/cf86d2ca271e4828ba8ce6cb78b2c2b4/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:51.323028 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36467fa-8bd2-4de8-b907-65ca1052dce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ef9812a3194dcb9c88e284be7e3982, user_id=cf86d2ca271e4828ba8ce6cb78b2c2b4, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:51.325800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36467fa-8bd2-4de8-b907-65ca1052dce1 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:51.453607 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1393/2734] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:51 2022] PUT /identity/v3/projects/23ef9812a3194dcb9c88e284be7e3982/users/cf86d2ca271e4828ba8ce6cb78b2c2b4/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:51.459115 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa18571-1c28-4e92-8b02-040102d6a7e4 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:51.460250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa18571-1c28-4e92-8b02-040102d6a7e4 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:51.460378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa18571-1c28-4e92-8b02-040102d6a7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:51.481234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-baa18571-1c28-4e92-8b02-040102d6a7e4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:51.482502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-baa18571-1c28-4e92-8b02-040102d6a7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:51.518498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:51.554602 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1342/2735] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:51.565744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36467fa-8bd2-4de8-b907-65ca1052dce1 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:51.567697 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-c36467fa-8bd2-4de8-b907-65ca1052dce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:51.636286 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36467fa-8bd2-4de8-b907-65ca1052dce1 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:51.640620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38879f47-f354-425a-b78c-2df5845724b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:51.641248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38879f47-f354-425a-b78c-2df5845724b3 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:51.641530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38879f47-f354-425a-b78c-2df5845724b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:51.641805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38879f47-f354-425a-b78c-2df5845724b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:51.653127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38879f47-f354-425a-b78c-2df5845724b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:51.656596 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38879f47-f354-425a-b78c-2df5845724b3 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:51.669261 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1394/2736] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:52.337714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa18571-1c28-4e92-8b02-040102d6a7e4 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:52.339396 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-baa18571-1c28-4e92-8b02-040102d6a7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:52.353637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa18571-1c28-4e92-8b02-040102d6a7e4 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:52.359163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:52.359853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:52.360102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:52.360428 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] PATH_INFO: `/v3/users/415973a355744b78a376ff5e2bbda1e8` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:52.361759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=415973a355744b78a376ff5e2bbda1e8)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:52.363730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:52.690784 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415973a355744b78a376ff5e2bbda1e8'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:52.708485 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415973a355744b78a376ff5e2bbda1e8'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:52.710306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415973a355744b78a376ff5e2bbda1e8'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:52.780477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415973a355744b78a376ff5e2bbda1e8'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:52.808992 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415973a355744b78a376ff5e2bbda1e8'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:52.817308 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1343/2737] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:52 2022] DELETE /identity/v3/users/415973a355744b78a376ff5e2bbda1e8 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:52.822755 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096333e-3797-416a-9f58-f00a387df37b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:52.823705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096333e-3797-416a-9f58-f00a387df37b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:52.823705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096333e-3797-416a-9f58-f00a387df37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:52.846117 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-5096333e-3797-416a-9f58-f00a387df37b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:52.846554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-5096333e-3797-416a-9f58-f00a387df37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:52.921485 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1395/2738] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:52.933068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:52.934620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:53.008810 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ea3ac-edbf-4a6d-8987-78edc07cc275 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:53.012772 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6208ae-5658-4e0e-bad1-b9cf7204e3a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:53.013445 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6208ae-5658-4e0e-bad1-b9cf7204e3a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:53.013734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6208ae-5658-4e0e-bad1-b9cf7204e3a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:53.014199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6208ae-5658-4e0e-bad1-b9cf7204e3a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:53.024844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6208ae-5658-4e0e-bad1-b9cf7204e3a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:53.026320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6208ae-5658-4e0e-bad1-b9cf7204e3a9 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:53.037788 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1344/2739] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:12:53.453359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf48dfb-0e31-40dc-8147-dac9ac759de0 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:53.454172 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf48dfb-0e31-40dc-8147-dac9ac759de0 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:53.454721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf48dfb-0e31-40dc-8147-dac9ac759de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:53.504095 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:53.528722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-abf48dfb-0e31-40dc-8147-dac9ac759de0 None None] MFA Rules not processed for user `cf86d2ca271e4828ba8ce6cb78b2c2b4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:53.538374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-abf48dfb-0e31-40dc-8147-dac9ac759de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:53.648598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1396/2740] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:53.667090 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6208ae-5658-4e0e-bad1-b9cf7204e3a9 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:53.668766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8c6208ae-5658-4e0e-bad1-b9cf7204e3a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:53.797215 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6208ae-5658-4e0e-bad1-b9cf7204e3a9 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:53.804133 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e836d-e70f-431a-bb2d-adb91a8d4e9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:53.804946 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e836d-e70f-431a-bb2d-adb91a8d4e9c service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:53.805313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e836d-e70f-431a-bb2d-adb91a8d4e9c service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:53.806378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e836d-e70f-431a-bb2d-adb91a8d4e9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:53.820905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e836d-e70f-431a-bb2d-adb91a8d4e9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:53.824874 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e836d-e70f-431a-bb2d-adb91a8d4e9c service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:53.834255 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:53.839683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1345/2741] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:12:54.105928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf48dfb-0e31-40dc-8147-dac9ac759de0 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:54.109297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-abf48dfb-0e31-40dc-8147-dac9ac759de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:54.145199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf48dfb-0e31-40dc-8147-dac9ac759de0 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:54.154904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:54.155894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:54.156810 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:54.157854 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] PATH_INFO: `/v3/projects/82f69fe7b30042b1a17aa3de8e613188` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:54.159166 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f69fe7b30042b1a17aa3de8e613188)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:54.161643 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:54.470543 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f69fe7b30042b1a17aa3de8e613188', 'request_id': 'req-93ad6534-311f-4da1-a378-03baf0aaf0db'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:54.490338 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f69fe7b30042b1a17aa3de8e613188', 'request_id': 'req-93ad6534-311f-4da1-a378-03baf0aaf0db'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:54.570501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f69fe7b30042b1a17aa3de8e613188', 'request_id': 'req-93ad6534-311f-4da1-a378-03baf0aaf0db'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:54.592030 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aa569-1d8e-4f6f-bfda-481d676707ed None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:54.592395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aa569-1d8e-4f6f-bfda-481d676707ed None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:54.592716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aa569-1d8e-4f6f-bfda-481d676707ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:54.595736 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] The token cache is being invalidate because project 82f69fe7b30042b1a17aa3de8e613188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:54.596275 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:54.597263 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:54.599628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1397/2742] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:54 2022] DELETE /identity/v3/projects/82f69fe7b30042b1a17aa3de8e613188 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:54.607324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:54.613670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:54.635405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-031aa569-1d8e-4f6f-bfda-481d676707ed None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:54.637273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-031aa569-1d8e-4f6f-bfda-481d676707ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:54.739629 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1346/2743] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:12:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:54.748150 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad6534-311f-4da1-a378-03baf0aaf0db admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:54.754915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:54.755788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:54.756141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:54.756617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031aa569-1d8e-4f6f-bfda-481d676707ed None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:54.756831 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] PATH_INFO: `/v3/users/f3515b3708a342fdb8dafe8cb7b6d1b1` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:54.758251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3515b3708a342fdb8dafe8cb7b6d1b1)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:54.758442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-031aa569-1d8e-4f6f-bfda-481d676707ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:54.760515 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:54.806514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:54.887842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031aa569-1d8e-4f6f-bfda-481d676707ed None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:54.894025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6c6d0-5582-4701-bf8d-7855840c965a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:54.894661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6c6d0-5582-4701-bf8d-7855840c965a service swift] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:54.894932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6c6d0-5582-4701-bf8d-7855840c965a service swift] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:54.895203 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6c6d0-5582-4701-bf8d-7855840c965a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:54.914118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6c6d0-5582-4701-bf8d-7855840c965a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:54.919066 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6c6d0-5582-4701-bf8d-7855840c965a service swift] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:54.927301 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1347/2744] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:12:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 12:12:55.019084 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3515b3708a342fdb8dafe8cb7b6d1b1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:55.019931 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3515b3708a342fdb8dafe8cb7b6d1b1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:55.029817 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3515b3708a342fdb8dafe8cb7b6d1b1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:55.035324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3515b3708a342fdb8dafe8cb7b6d1b1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:55.113266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-83615582-4e1b-4d13-85b3-d5fe22eea799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3515b3708a342fdb8dafe8cb7b6d1b1'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:55.153003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1398/2745] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:54 2022] DELETE /identity/v3/users/f3515b3708a342fdb8dafe8cb7b6d1b1 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:55.874620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6c6d0-5582-4701-bf8d-7855840c965a service swift] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:55.917649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6c6d0-5582-4701-bf8d-7855840c965a service swift] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:56.111592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:56.111892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:56.112269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:56.112723 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] PATH_INFO: `/v3/projects/8627cf2f447740ddaf86afe796929be0` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:56.114850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8627cf2f447740ddaf86afe796929be0)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:56.116097 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34a92b-f097-4403-957b-e2bd9d7af991 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:56.116729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34a92b-f097-4403-957b-e2bd9d7af991 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:56.119995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34a92b-f097-4403-957b-e2bd9d7af991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:56.127447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:56.151590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-2b34a92b-f097-4403-957b-e2bd9d7af991 None None] MFA Rules not processed for user `cf86d2ca271e4828ba8ce6cb78b2c2b4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:56.154220 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2b34a92b-f097-4403-957b-e2bd9d7af991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:56.265766 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1399/2746] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:56.319976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:56.334791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8627cf2f447740ddaf86afe796929be0', 'request_id': 'req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:56.343875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8627cf2f447740ddaf86afe796929be0', 'request_id': 'req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:56.353873 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8627cf2f447740ddaf86afe796929be0', 'request_id': 'req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:56.460359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] The token cache is being invalidate because project 8627cf2f447740ddaf86afe796929be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:56.461282 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:56.462898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:56.469813 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1348/2747] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:55 2022] DELETE /identity/v3/projects/8627cf2f447740ddaf86afe796929be0 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:56.476739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34a92b-f097-4403-957b-e2bd9d7af991 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:56.479062 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-2b34a92b-f097-4403-957b-e2bd9d7af991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:56.515079 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34a92b-f097-4403-957b-e2bd9d7af991 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:56.631182 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:56.632059 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:56.633860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:56.633860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] PATH_INFO: `/v3/users/472f15d601da4469b56ee23ba42e1d53` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:56.634358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=472f15d601da4469b56ee23ba42e1d53)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:56.636764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:56.930227 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '472f15d601da4469b56ee23ba42e1d53'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:56.931351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '472f15d601da4469b56ee23ba42e1d53'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:56.946146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:56.950614 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '472f15d601da4469b56ee23ba42e1d53'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:56.952154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '472f15d601da4469b56ee23ba42e1d53'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:57.049958 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e9f8199d-18d0-43a4-88b7-014307a8f909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '472f15d601da4469b56ee23ba42e1d53'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:57.092991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1400/2748] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:12:56 2022] DELETE /identity/v3/users/472f15d601da4469b56ee23ba42e1d53 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:57.914580 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:57.917464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:57.952325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e11adf-49a7-4ef0-a8a3-8de4703896c5 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:58.041853 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:58.042723 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:58.043091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:58.043472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] PATH_INFO: `/v3/projects/4e2e2a23f0b04304842ed410fa48ac1f` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:58.044869 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e2e2a23f0b04304842ed410fa48ac1f)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:58.046959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:58.275847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e2e2a23f0b04304842ed410fa48ac1f', 'request_id': 'req-f59033e2-a7a7-46e7-bf1e-acff043d8240'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:58.286112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e2e2a23f0b04304842ed410fa48ac1f', 'request_id': 'req-f59033e2-a7a7-46e7-bf1e-acff043d8240'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:58.295955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e2e2a23f0b04304842ed410fa48ac1f', 'request_id': 'req-f59033e2-a7a7-46e7-bf1e-acff043d8240'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:58.487414 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] The token cache is being invalidate because project 4e2e2a23f0b04304842ed410fa48ac1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:12:58.487718 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:58.488470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-f59033e2-a7a7-46e7-bf1e-acff043d8240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:12:58.491025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1349/2749] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:12:57 2022] DELETE /identity/v3/projects/4e2e2a23f0b04304842ed410fa48ac1f => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:12:58.498247 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d838c3-da70-4f87-8abe-80da31baa92b None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:58.499208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d838c3-da70-4f87-8abe-80da31baa92b None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:58.499208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d838c3-da70-4f87-8abe-80da31baa92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:58.524256 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-e1d838c3-da70-4f87-8abe-80da31baa92b None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:58.526464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e1d838c3-da70-4f87-8abe-80da31baa92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:58.637761 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1401/2750] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:12:58.640422 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd9960-f745-4a3b-81c4-fb42af6b07d7 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:58.640894 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd9960-f745-4a3b-81c4-fb42af6b07d7 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:58.641362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd9960-f745-4a3b-81c4-fb42af6b07d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:58.669701 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-0cbd9960-f745-4a3b-81c4-fb42af6b07d7 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:58.670969 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0cbd9960-f745-4a3b-81c4-fb42af6b07d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:58.684623 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1350/2751] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:58.686910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d838c3-da70-4f87-8abe-80da31baa92b None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:58.703983 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d838c3-da70-4f87-8abe-80da31baa92b None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:58.718491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b3a85-82b7-4bc3-bc49-e783c9cd1674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:58.718491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b3a85-82b7-4bc3-bc49-e783c9cd1674 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:58.718491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b3a85-82b7-4bc3-bc49-e783c9cd1674 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:58.718491 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b3a85-82b7-4bc3-bc49-e783c9cd1674 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:58.719968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b3a85-82b7-4bc3-bc49-e783c9cd1674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:12:58.719968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b3a85-82b7-4bc3-bc49-e783c9cd1674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:58.719968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b3a85-82b7-4bc3-bc49-e783c9cd1674 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:58.742404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1402/2752] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:12:58.744306 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd9960-f745-4a3b-81c4-fb42af6b07d7 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:58.759755 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd9960-f745-4a3b-81c4-fb42af6b07d7 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:58.769189 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70a89b-f84b-446a-87fe-d30f3ffbf3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:58.770587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70a89b-f84b-446a-87fe-d30f3ffbf3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:58.771348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70a89b-f84b-446a-87fe-d30f3ffbf3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:58.772044 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70a89b-f84b-446a-87fe-d30f3ffbf3e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:58.773535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70a89b-f84b-446a-87fe-d30f3ffbf3e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:58.777711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70a89b-f84b-446a-87fe-d30f3ffbf3e3 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:58.920975 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1351/2753] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:12:58 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:58.925351 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b3a85-82b7-4bc3-bc49-e783c9cd1674 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:58.938751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b3a85-82b7-4bc3-bc49-e783c9cd1674 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:59.018902 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d5a68-65fa-41de-8001-56bff771161f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:59.019538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d5a68-65fa-41de-8001-56bff771161f admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:59.019878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d5a68-65fa-41de-8001-56bff771161f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:59.020160 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d5a68-65fa-41de-8001-56bff771161f admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:59.020870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d5a68-65fa-41de-8001-56bff771161f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:59.023151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d5a68-65fa-41de-8001-56bff771161f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:59.107151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1403/2754] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:12:58 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:12:59.111618 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70a89b-f84b-446a-87fe-d30f3ffbf3e3 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:59.133688 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70a89b-f84b-446a-87fe-d30f3ffbf3e3 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:59.140602 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7d6a7-2859-4250-b56e-3712aacd9685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:59.141474 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7d6a7-2859-4250-b56e-3712aacd9685 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:59.141883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7d6a7-2859-4250-b56e-3712aacd9685 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:59.142277 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7d6a7-2859-4250-b56e-3712aacd9685 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:59.143235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7d6a7-2859-4250-b56e-3712aacd9685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:59.145782 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7d6a7-2859-4250-b56e-3712aacd9685 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:59.163193 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1352/2755] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:12:59.167478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d5a68-65fa-41de-8001-56bff771161f admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:59.179215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d5a68-65fa-41de-8001-56bff771161f admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:59.184132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b21a7a-b391-4ace-b178-aff5a0c4b1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:59.184774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b21a7a-b391-4ace-b178-aff5a0c4b1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:59.185056 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b21a7a-b391-4ace-b178-aff5a0c4b1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:59.185333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b21a7a-b391-4ace-b178-aff5a0c4b1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:59.185980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b21a7a-b391-4ace-b178-aff5a0c4b1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:59.187589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b21a7a-b391-4ace-b178-aff5a0c4b1c5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:59.201115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1404/2756] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:12:59.207520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7d6a7-2859-4250-b56e-3712aacd9685 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:59.218898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7d6a7-2859-4250-b56e-3712aacd9685 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:59.227211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de51d5-abe4-4902-9779-f0e3d3c17961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:59.228367 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de51d5-abe4-4902-9779-f0e3d3c17961 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:59.228841 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de51d5-abe4-4902-9779-f0e3d3c17961 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:59.229354 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de51d5-abe4-4902-9779-f0e3d3c17961 admin admin] PATH_INFO: `/v3/projects/f8feaa0c5e9349cc8e2dcc3d59120aa9/users/1f685bdbb59f4d4b9eb68e82beedf195/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:59.245378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:59.249874 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de51d5-abe4-4902-9779-f0e3d3c17961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8feaa0c5e9349cc8e2dcc3d59120aa9, user_id=1f685bdbb59f4d4b9eb68e82beedf195, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:59.263392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de51d5-abe4-4902-9779-f0e3d3c17961 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:59.416423 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1353/2757] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:12:59 2022] PUT /identity/v3/projects/f8feaa0c5e9349cc8e2dcc3d59120aa9/users/1f685bdbb59f4d4b9eb68e82beedf195/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:12:59.437155 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde45fb-32cb-4972-ac40-ea8865111a61 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:59.437468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde45fb-32cb-4972-ac40-ea8865111a61 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:59.437744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde45fb-32cb-4972-ac40-ea8865111a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:59.477387 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-1fde45fb-32cb-4972-ac40-ea8865111a61 None None] MFA Rules not processed for user `1f685bdbb59f4d4b9eb68e82beedf195`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:12:59.479022 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-1fde45fb-32cb-4972-ac40-ea8865111a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:59.506859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:12:59.585716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1405/2758] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:12:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:12:59.603659 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de51d5-abe4-4902-9779-f0e3d3c17961 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:12:59.607268 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-93de51d5-abe4-4902-9779-f0e3d3c17961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:12:59.701598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de51d5-abe4-4902-9779-f0e3d3c17961 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:12:59.708023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c2f27-e115-4ad0-88cd-fd027af8aaa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:12:59.709700 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c2f27-e115-4ad0-88cd-fd027af8aaa0 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:12:59.710324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c2f27-e115-4ad0-88cd-fd027af8aaa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:12:59.710868 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c2f27-e115-4ad0-88cd-fd027af8aaa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:12:59.729809 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c2f27-e115-4ad0-88cd-fd027af8aaa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:12:59.733070 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c2f27-e115-4ad0-88cd-fd027af8aaa0 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:12:59.754790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1354/2759] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:12:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:13:01.790825 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cfee-2744-48e0-b4d3-eafdf0249f53 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:01.791390 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cfee-2744-48e0-b4d3-eafdf0249f53 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:01.791816 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cfee-2744-48e0-b4d3-eafdf0249f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:01.827419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-10b7cfee-2744-48e0-b4d3-eafdf0249f53 None None] MFA Rules not processed for user `1f685bdbb59f4d4b9eb68e82beedf195`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:13:01.829811 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-10b7cfee-2744-48e0-b4d3-eafdf0249f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:01.856762 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1406/2760] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:13:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:13:09.745612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:13:10.949797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c2f27-e115-4ad0-88cd-fd027af8aaa0 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:13:10.951290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-be2c2f27-e115-4ad0-88cd-fd027af8aaa0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:11.049502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c2f27-e115-4ad0-88cd-fd027af8aaa0 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:13:11.057127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7797d1-b6ef-42db-96cb-4a2e94fb6f3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:13:11.058293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7797d1-b6ef-42db-96cb-4a2e94fb6f3f service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:11.058951 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7797d1-b6ef-42db-96cb-4a2e94fb6f3f service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:11.059568 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7797d1-b6ef-42db-96cb-4a2e94fb6f3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:11.063148 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ee7797d1-b6ef-42db-96cb-4a2e94fb6f3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:11.083392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7797d1-b6ef-42db-96cb-4a2e94fb6f3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:13:11.087025 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7797d1-b6ef-42db-96cb-4a2e94fb6f3f service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:13:11.100962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1355/2761] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:13:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:13:11.817910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=21 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:13:19.680885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb803-d1cf-4f33-a17d-8d292e29905a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:19.681578 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb803-d1cf-4f33-a17d-8d292e29905a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:19.682033 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb803-d1cf-4f33-a17d-8d292e29905a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:19.748588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-36eeb803-d1cf-4f33-a17d-8d292e29905a None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:13:19.749883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-36eeb803-d1cf-4f33-a17d-8d292e29905a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:19.826285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1407/2762] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 4 12:13:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:13:19.841790 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7797d1-b6ef-42db-96cb-4a2e94fb6f3f service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:13:19.843320 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-ee7797d1-b6ef-42db-96cb-4a2e94fb6f3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:19.955209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7797d1-b6ef-42db-96cb-4a2e94fb6f3f service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:13:19.963065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acd875-2301-4389-844e-299f7663495d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:13:19.964085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acd875-2301-4389-844e-299f7663495d service swift] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:19.964626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acd875-2301-4389-844e-299f7663495d service swift] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:19.965276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acd875-2301-4389-844e-299f7663495d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:19.979976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acd875-2301-4389-844e-299f7663495d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:13:19.982163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acd875-2301-4389-844e-299f7663495d service swift] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:13:19.994007 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1356/2763] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:13:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:13:27.839067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb803-d1cf-4f33-a17d-8d292e29905a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:13:27.842393 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-36eeb803-d1cf-4f33-a17d-8d292e29905a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:27.945866 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb803-d1cf-4f33-a17d-8d292e29905a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:13:27.954278 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ebbc4-5910-4821-9b23-4a311ac19d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:13:27.955180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ebbc4-5910-4821-9b23-4a311ac19d60 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:27.955710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ebbc4-5910-4821-9b23-4a311ac19d60 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:27.956201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ebbc4-5910-4821-9b23-4a311ac19d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:27.960478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-476ebbc4-5910-4821-9b23-4a311ac19d60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:27.983833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ebbc4-5910-4821-9b23-4a311ac19d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:13:27.986310 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ebbc4-5910-4821-9b23-4a311ac19d60 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:13:27.999141 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1408/2764] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:13:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:13:29.990009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=67 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:13:30.353875 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5817ca-100a-4507-a09d-98da3fca1da5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:30.354130 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5817ca-100a-4507-a09d-98da3fca1da5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:30.354410 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5817ca-100a-4507-a09d-98da3fca1da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:30.376068 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-af5817ca-100a-4507-a09d-98da3fca1da5 None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:13:30.377705 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-af5817ca-100a-4507-a09d-98da3fca1da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:30.386727 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1357/2765] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:13:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:13:30.409233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ebbc4-5910-4821-9b23-4a311ac19d60 service nova] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:13:30.420715 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ebbc4-5910-4821-9b23-4a311ac19d60 service nova] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:13:30.427386 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c1fe9-db00-49bb-85f6-db1c3260458c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:13:30.428002 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1fe9-db00-49bb-85f6-db1c3260458c service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:30.428419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1fe9-db00-49bb-85f6-db1c3260458c service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:30.428726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1fe9-db00-49bb-85f6-db1c3260458c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:30.438274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c1fe9-db00-49bb-85f6-db1c3260458c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:13:30.440098 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c1fe9-db00-49bb-85f6-db1c3260458c service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:13:30.450626 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1409/2766] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:13:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:13:31.704492 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4b763-c49c-420b-87dd-da2559116d4e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:31.705037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4b763-c49c-420b-87dd-da2559116d4e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:31.705037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4b763-c49c-420b-87dd-da2559116d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:31.726912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-e3a4b763-c49c-420b-87dd-da2559116d4e None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:13:31.728231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e3a4b763-c49c-420b-87dd-da2559116d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:31.737074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1358/2767] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 4 12:13:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:13:31.754337 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c1fe9-db00-49bb-85f6-db1c3260458c service swift] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:13:31.766768 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c1fe9-db00-49bb-85f6-db1c3260458c service swift] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:13:31.773090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b1594-44b9-4858-b6a4-1b6a134197e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:13:31.773781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b1594-44b9-4858-b6a4-1b6a134197e3 service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:31.774119 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b1594-44b9-4858-b6a4-1b6a134197e3 service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:31.774396 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b1594-44b9-4858-b6a4-1b6a134197e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:31.791326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b1594-44b9-4858-b6a4-1b6a134197e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:13:31.793554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b1594-44b9-4858-b6a4-1b6a134197e3 service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:13:31.807946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1410/2768] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:13:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 12:13:32.615450 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd96c68-3f24-4c8b-ad14-649a1cd82b93 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:32.618727 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd96c68-3f24-4c8b-ad14-649a1cd82b93 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:32.618727 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd96c68-3f24-4c8b-ad14-649a1cd82b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:32.638667 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-dbd96c68-3f24-4c8b-ad14-649a1cd82b93 None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:13:32.640266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-dbd96c68-3f24-4c8b-ad14-649a1cd82b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:32.648735 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1359/2769] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:13:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:13:32.662798 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b1594-44b9-4858-b6a4-1b6a134197e3 service swift] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:13:32.677729 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b1594-44b9-4858-b6a4-1b6a134197e3 service swift] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:13:32.684001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca580a-9813-4a61-848a-5f018528ec60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:13:32.684668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca580a-9813-4a61-848a-5f018528ec60 service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:32.684941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca580a-9813-4a61-848a-5f018528ec60 service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:32.685215 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca580a-9813-4a61-848a-5f018528ec60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:32.695558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca580a-9813-4a61-848a-5f018528ec60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:13:32.697367 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca580a-9813-4a61-848a-5f018528ec60 service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:13:32.707499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1411/2770] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:13:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 12:13:33.490847 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cae2a-44d9-4174-8b49-608333aa4647 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:33.491539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cae2a-44d9-4174-8b49-608333aa4647 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:33.491539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cae2a-44d9-4174-8b49-608333aa4647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:33.513674 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-a73cae2a-44d9-4174-8b49-608333aa4647 None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:13:33.514953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a73cae2a-44d9-4174-8b49-608333aa4647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:33.523206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1360/2771] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 4 12:13:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:13:33.537507 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca580a-9813-4a61-848a-5f018528ec60 service swift] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:13:33.557162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca580a-9813-4a61-848a-5f018528ec60 service swift] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:13:33.566290 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8683a3-3678-4cce-a120-18848ab22922 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:13:33.567530 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8683a3-3678-4cce-a120-18848ab22922 service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:33.568045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8683a3-3678-4cce-a120-18848ab22922 service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:33.568588 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8683a3-3678-4cce-a120-18848ab22922 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:33.583995 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8683a3-3678-4cce-a120-18848ab22922 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:13:33.586775 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8683a3-3678-4cce-a120-18848ab22922 service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:13:33.603108 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1412/2772] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:13:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:13:36.630909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cae2a-44d9-4174-8b49-608333aa4647 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:13:36.729205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cae2a-44d9-4174-8b49-608333aa4647 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:13:36.736973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:13:36.738091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:36.738571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:36.739079 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] PATH_INFO: `/v3/users/1f685bdbb59f4d4b9eb68e82beedf195` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:36.741235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f685bdbb59f4d4b9eb68e82beedf195)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:13:36.745064 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:13:36.938958 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f685bdbb59f4d4b9eb68e82beedf195'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:13:36.948476 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f685bdbb59f4d4b9eb68e82beedf195'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:13:36.949425 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f685bdbb59f4d4b9eb68e82beedf195'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:13:37.046236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f685bdbb59f4d4b9eb68e82beedf195'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:13:37.063225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f685bdbb59f4d4b9eb68e82beedf195'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:13:37.071481 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1361/2773] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:13:36 2022] DELETE /identity/v3/users/1f685bdbb59f4d4b9eb68e82beedf195 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:13:37.079342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09079055-32e6-4fb5-b4e3-dc1711d3c6fb None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:37.080324 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09079055-32e6-4fb5-b4e3-dc1711d3c6fb None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:37.080552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09079055-32e6-4fb5-b4e3-dc1711d3c6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:37.091550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=100 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:13:37.104920 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-09079055-32e6-4fb5-b4e3-dc1711d3c6fb None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:13:37.106915 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-09079055-32e6-4fb5-b4e3-dc1711d3c6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:37.198012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1413/2774] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:13:37.209747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:13:37.211800 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:37.285561 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd34c3-d941-4ae7-b5da-3f898725338d admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:13:37.290680 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533101e-05e1-4e68-9392-5903a27ab906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:13:37.291447 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533101e-05e1-4e68-9392-5903a27ab906 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:37.291848 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533101e-05e1-4e68-9392-5903a27ab906 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:37.292034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533101e-05e1-4e68-9392-5903a27ab906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:37.300903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:13:37.304639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533101e-05e1-4e68-9392-5903a27ab906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:13:37.306758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533101e-05e1-4e68-9392-5903a27ab906 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:13:37.319037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1362/2775] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:13:37.661477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09079055-32e6-4fb5-b4e3-dc1711d3c6fb None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:13:37.663246 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-09079055-32e6-4fb5-b4e3-dc1711d3c6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:13:37.709451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09079055-32e6-4fb5-b4e3-dc1711d3c6fb None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:13:37.718128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:13:37.720222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:13:37.720222 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:13:37.720972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] PATH_INFO: `/v3/projects/f8feaa0c5e9349cc8e2dcc3d59120aa9` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:13:37.727066 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8feaa0c5e9349cc8e2dcc3d59120aa9)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:13:37.731264 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:13:37.889464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8feaa0c5e9349cc8e2dcc3d59120aa9', 'request_id': 'req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:13:37.897099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8feaa0c5e9349cc8e2dcc3d59120aa9', 'request_id': 'req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:13:37.994801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8feaa0c5e9349cc8e2dcc3d59120aa9', 'request_id': 'req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:13:38.011239 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] The token cache is being invalidate because project f8feaa0c5e9349cc8e2dcc3d59120aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:13:38.011670 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:13:38.012573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:13:38.014546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1414/2776] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:13:37 2022] DELETE /identity/v3/projects/f8feaa0c5e9349cc8e2dcc3d59120aa9 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:13:47.571437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:13:48.007663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:14:27.908312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c636006-f1d2-45d9-bb62-ae5fd0086327 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:14:27.908976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c636006-f1d2-45d9-bb62-ae5fd0086327 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:14:27.908976 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c636006-f1d2-45d9-bb62-ae5fd0086327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:14:27.929844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-8c636006-f1d2-45d9-bb62-ae5fd0086327 None None] MFA Rules not processed for user `9ab97a61223f4719add7b3c0f845a319`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:14:27.931085 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8c636006-f1d2-45d9-bb62-ae5fd0086327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:28.030197 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1363/2777] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:14:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:14:28.395859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:14:28.398319 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:28.483779 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd4647-5fe2-471f-a6aa-dd78afa6e59e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:14:28.488111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ae1c8-2d9b-4951-a906-4e0f03d93b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:14:28.488784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ae1c8-2d9b-4951-a906-4e0f03d93b05 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:14:28.489063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ae1c8-2d9b-4951-a906-4e0f03d93b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:14:28.489333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ae1c8-2d9b-4951-a906-4e0f03d93b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:14:28.491214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-158ae1c8-2d9b-4951-a906-4e0f03d93b05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:28.502645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ae1c8-2d9b-4951-a906-4e0f03d93b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:14:28.504094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ae1c8-2d9b-4951-a906-4e0f03d93b05 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:14:28.513276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1415/2778] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:14:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:14:29.015986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c636006-f1d2-45d9-bb62-ae5fd0086327 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:14:29.015986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8c636006-f1d2-45d9-bb62-ae5fd0086327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:29.121934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c636006-f1d2-45d9-bb62-ae5fd0086327 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:14:29.130202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3b102-4929-49b7-9906-2e3bda073e26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:14:29.133339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3b102-4929-49b7-9906-2e3bda073e26 service placement] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:14:29.133339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3b102-4929-49b7-9906-2e3bda073e26 service placement] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:14:29.133339 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3b102-4929-49b7-9906-2e3bda073e26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:14:29.133779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3af3b102-4929-49b7-9906-2e3bda073e26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:29.149040 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3b102-4929-49b7-9906-2e3bda073e26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:14:29.150962 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3b102-4929-49b7-9906-2e3bda073e26 service placement] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:14:29.163295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1364/2779] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:14:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:14:38.509235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=26 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:14:39.158840 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=57 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:14:40.004887 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086286f-1dfa-4b90-860b-64b8e8a5a09f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:14:40.005786 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086286f-1dfa-4b90-860b-64b8e8a5a09f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:14:40.005786 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086286f-1dfa-4b90-860b-64b8e8a5a09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:14:40.050545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-4086286f-1dfa-4b90-860b-64b8e8a5a09f None None] MFA Rules not processed for user `cf86d2ca271e4828ba8ce6cb78b2c2b4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:14:40.050545 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-4086286f-1dfa-4b90-860b-64b8e8a5a09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:40.211068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1416/2780] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:14:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:14:41.358286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3b102-4929-49b7-9906-2e3bda073e26 service placement] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:14:41.361627 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3af3b102-4929-49b7-9906-2e3bda073e26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:41.503917 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3b102-4929-49b7-9906-2e3bda073e26 service placement] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:14:41.510359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523ba39-cb07-4db7-a25d-998b8bd41877 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:14:41.510986 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523ba39-cb07-4db7-a25d-998b8bd41877 service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:14:41.511424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523ba39-cb07-4db7-a25d-998b8bd41877 service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:14:41.511648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523ba39-cb07-4db7-a25d-998b8bd41877 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:14:41.513981 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f523ba39-cb07-4db7-a25d-998b8bd41877 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:41.530709 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523ba39-cb07-4db7-a25d-998b8bd41877 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:14:41.533427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523ba39-cb07-4db7-a25d-998b8bd41877 service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:14:41.548213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1365/2781] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:14:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:14:46.280710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086286f-1dfa-4b90-860b-64b8e8a5a09f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:14:46.292605 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086286f-1dfa-4b90-860b-64b8e8a5a09f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:14:46.300123 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73d1b6-b0e5-45c5-8082-9e10b57019c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:14:46.300911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73d1b6-b0e5-45c5-8082-9e10b57019c1 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:14:46.301270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73d1b6-b0e5-45c5-8082-9e10b57019c1 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:14:46.301653 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73d1b6-b0e5-45c5-8082-9e10b57019c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:14:46.304042 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cd73d1b6-b0e5-45c5-8082-9e10b57019c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:46.319419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73d1b6-b0e5-45c5-8082-9e10b57019c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:14:46.321202 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73d1b6-b0e5-45c5-8082-9e10b57019c1 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:14:46.336218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1417/2782] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:14:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:14:51.543948 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=34 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:14:56.328835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=31 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:14:59.404381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523ba39-cb07-4db7-a25d-998b8bd41877 service nova] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:14:59.406381 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f523ba39-cb07-4db7-a25d-998b8bd41877 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:59.421842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523ba39-cb07-4db7-a25d-998b8bd41877 service nova] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:14:59.428760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6b660-5faa-46f0-b27d-fdab75f4462a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:14:59.429654 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6b660-5faa-46f0-b27d-fdab75f4462a service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:14:59.430087 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6b660-5faa-46f0-b27d-fdab75f4462a service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:14:59.430581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6b660-5faa-46f0-b27d-fdab75f4462a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:14:59.433931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-73a6b660-5faa-46f0-b27d-fdab75f4462a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:14:59.527588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6b660-5faa-46f0-b27d-fdab75f4462a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:14:59.529517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6b660-5faa-46f0-b27d-fdab75f4462a service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:14:59.542318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1366/2783] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:15:03.872439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73d1b6-b0e5-45c5-8082-9e10b57019c1 service nova] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:15:03.896391 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73d1b6-b0e5-45c5-8082-9e10b57019c1 service nova] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:15:03.914716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7447732-8783-416e-bc31-8d406f9730dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:15:03.914716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7447732-8783-416e-bc31-8d406f9730dc service placement] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:15:03.914716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7447732-8783-416e-bc31-8d406f9730dc service placement] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:15:03.914716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7447732-8783-416e-bc31-8d406f9730dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:15:03.918538 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f7447732-8783-416e-bc31-8d406f9730dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:15:03.933947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7447732-8783-416e-bc31-8d406f9730dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:15:03.935678 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7447732-8783-416e-bc31-8d406f9730dc service placement] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:15:03.948952 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1418/2784] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:15:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:15:08.870264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6b660-5faa-46f0-b27d-fdab75f4462a service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:15:08.872972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-73a6b660-5faa-46f0-b27d-fdab75f4462a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:15:08.891858 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6b660-5faa-46f0-b27d-fdab75f4462a service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:15:08.897994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:15:08.898795 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:15:08.899136 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:15:08.899467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] PATH_INFO: `/v3/users/76771369a0df4c5189e42291ebdf9d24` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:15:08.916787 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] RBAC: Authorizing `identity:delete_user(user_id=76771369a0df4c5189e42291ebdf9d24)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:15:08.918747 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:15:09.131122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76771369a0df4c5189e42291ebdf9d24'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:09.140150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76771369a0df4c5189e42291ebdf9d24'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:09.141572 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76771369a0df4c5189e42291ebdf9d24'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:09.216904 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76771369a0df4c5189e42291ebdf9d24'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:09.232619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76771369a0df4c5189e42291ebdf9d24'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:09.240127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1367/2785] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:15:08 2022] DELETE /identity/v3/users/76771369a0df4c5189e42291ebdf9d24 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:15:09.511537 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7447732-8783-416e-bc31-8d406f9730dc service placement] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:15:09.527291 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7447732-8783-416e-bc31-8d406f9730dc service placement] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:15:09.598008 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:15:09.598990 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:15:09.599398 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:15:09.599721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] PATH_INFO: `/v3/projects/d4da0e2bff704d7a944445d0011a8667` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:15:09.612232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4da0e2bff704d7a944445d0011a8667)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:15:09.614245 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:15:09.735512 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4da0e2bff704d7a944445d0011a8667', 'request_id': 'req-6d575261-e60e-4997-8d2e-c08b339e9bd7'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:09.741954 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4da0e2bff704d7a944445d0011a8667', 'request_id': 'req-6d575261-e60e-4997-8d2e-c08b339e9bd7'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:09.834471 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4da0e2bff704d7a944445d0011a8667', 'request_id': 'req-6d575261-e60e-4997-8d2e-c08b339e9bd7'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:09.856433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] The token cache is being invalidate because project d4da0e2bff704d7a944445d0011a8667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:15:09.856879 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:09.857698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:09.859772 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1419/2786] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:15:09 2022] DELETE /identity/v3/projects/d4da0e2bff704d7a944445d0011a8667 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:15:09.864293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:15:09.866187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:15:09.883454 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb05708-deca-421c-b90e-2cc129db666c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:15:09.958571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:15:09.969966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:15:09.971120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:15:09.971614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:15:09.972109 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] PATH_INFO: `/v3/users/e133231fd3264083a2184b9685f212d9` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:15:09.999636 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e133231fd3264083a2184b9685f212d9)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:15:10.004533 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:15:10.263443 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e133231fd3264083a2184b9685f212d9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:10.272318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e133231fd3264083a2184b9685f212d9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:10.273204 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e133231fd3264083a2184b9685f212d9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:10.346921 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e133231fd3264083a2184b9685f212d9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:10.363248 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e133231fd3264083a2184b9685f212d9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:10.370487 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1368/2787] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:15:09 2022] DELETE /identity/v3/users/e133231fd3264083a2184b9685f212d9 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:15:10.696250 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:15:10.698013 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:15:10.715242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d575261-e60e-4997-8d2e-c08b339e9bd7 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:15:10.792439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:15:10.793032 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:15:10.793367 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:15:10.793589 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] PATH_INFO: `/v3/projects/967915d2bb87499d9b15d7f694f76e67` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:15:10.813940 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=967915d2bb87499d9b15d7f694f76e67)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:15:10.817844 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:15:10.820753 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:15:11.002341 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967915d2bb87499d9b15d7f694f76e67', 'request_id': 'req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:11.012478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967915d2bb87499d9b15d7f694f76e67', 'request_id': 'req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:11.164524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967915d2bb87499d9b15d7f694f76e67', 'request_id': 'req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:11.179886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] The token cache is being invalidate because project 967915d2bb87499d9b15d7f694f76e67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:15:11.180228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:11.180972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:15:11.182551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1420/2788] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:15:10 2022] DELETE /identity/v3/projects/967915d2bb87499d9b15d7f694f76e67 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:15:20.366190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:15:21.326512 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:15:40.624793 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:15:40.626913 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:15:40.716882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03559d9b-a154-4d8c-b305-4a4cd0c20c88 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:15:40.801605 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e5ec7-5efe-4851-af6b-e07903f4e5c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:15:40.802613 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e5ec7-5efe-4851-af6b-e07903f4e5c9 service placement] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:15:40.803076 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e5ec7-5efe-4851-af6b-e07903f4e5c9 service placement] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:15:40.803705 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e5ec7-5efe-4851-af6b-e07903f4e5c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:15:40.806257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c90e5ec7-5efe-4851-af6b-e07903f4e5c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:15:40.819307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e5ec7-5efe-4851-af6b-e07903f4e5c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:15:40.821183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e5ec7-5efe-4851-af6b-e07903f4e5c9 service placement] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:15:40.835589 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1369/2789] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:15:50.829113 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=58 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:15:51.265115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:15:51.267938 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:15:51.358880 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5300eb-6a95-4bb7-bdc8-dd51f8a22b53 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:15:51.365173 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768cc43-e75f-4e98-b73b-3c883024c238 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:15:51.365909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768cc43-e75f-4e98-b73b-3c883024c238 service nova] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:15:51.366205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768cc43-e75f-4e98-b73b-3c883024c238 service nova] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:15:51.366514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768cc43-e75f-4e98-b73b-3c883024c238 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:15:51.368548 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3768cc43-e75f-4e98-b73b-3c883024c238 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:15:51.432077 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768cc43-e75f-4e98-b73b-3c883024c238 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:15:51.433682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768cc43-e75f-4e98-b73b-3c883024c238 service nova] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:15:51.443693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1421/2790] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:15:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:16:00.486731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e5ec7-5efe-4851-af6b-e07903f4e5c9 service placement] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:16:00.488532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c90e5ec7-5efe-4851-af6b-e07903f4e5c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:00.507477 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e5ec7-5efe-4851-af6b-e07903f4e5c9 service placement] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:16:00.515417 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aa6b8-76d7-479f-b3ac-94dec780ded0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:16:00.516571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa6b8-76d7-479f-b3ac-94dec780ded0 service placement] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:00.518008 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa6b8-76d7-479f-b3ac-94dec780ded0 service placement] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:00.518526 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa6b8-76d7-479f-b3ac-94dec780ded0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:00.521506 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-a63aa6b8-76d7-479f-b3ac-94dec780ded0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:00.539017 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aa6b8-76d7-479f-b3ac-94dec780ded0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:16:00.541186 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aa6b8-76d7-479f-b3ac-94dec780ded0 service placement] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:16:00.556489 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1370/2791] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:16:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:16:00.689455 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768cc43-e75f-4e98-b73b-3c883024c238 service nova] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:16:00.691781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-3768cc43-e75f-4e98-b73b-3c883024c238 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:00.810057 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768cc43-e75f-4e98-b73b-3c883024c238 service nova] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:16:00.818623 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:16:00.820017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:00.820656 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:00.821151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] PATH_INFO: `/v3/users/cf86d2ca271e4828ba8ce6cb78b2c2b4` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:00.823301 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf86d2ca271e4828ba8ce6cb78b2c2b4)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:16:00.827325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:16:01.094663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:16:01.105016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf86d2ca271e4828ba8ce6cb78b2c2b4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:16:01.105016 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf86d2ca271e4828ba8ce6cb78b2c2b4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:16:01.112325 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf86d2ca271e4828ba8ce6cb78b2c2b4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:16:01.113266 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf86d2ca271e4828ba8ce6cb78b2c2b4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:16:01.186905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-a324ad30-986c-4565-af71-17f3dd091a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf86d2ca271e4828ba8ce6cb78b2c2b4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:16:01.208972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1422/2792] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:16:00 2022] DELETE /identity/v3/users/cf86d2ca271e4828ba8ce6cb78b2c2b4 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:16:01.626535 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aa6b8-76d7-479f-b3ac-94dec780ded0 service placement] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:16:01.642994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aa6b8-76d7-479f-b3ac-94dec780ded0 service placement] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:16:01.735328 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:16:01.736164 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:01.736582 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:01.736955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] PATH_INFO: `/v3/projects/23ef9812a3194dcb9c88e284be7e3982` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:01.738427 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ef9812a3194dcb9c88e284be7e3982)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:16:01.740816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:16:01.895433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ef9812a3194dcb9c88e284be7e3982', 'request_id': 'req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:16:01.903074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ef9812a3194dcb9c88e284be7e3982', 'request_id': 'req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:16:01.911298 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ef9812a3194dcb9c88e284be7e3982', 'request_id': 'req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:16:02.022440 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] The token cache is being invalidate because project 23ef9812a3194dcb9c88e284be7e3982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:16:02.022964 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:16:02.023767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:16:02.025463 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1371/2793] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:16:01 2022] DELETE /identity/v3/projects/23ef9812a3194dcb9c88e284be7e3982 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:16:07.331837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b433f-ae47-4f08-9839-dbeec6a5357a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:07.332492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b433f-ae47-4f08-9839-dbeec6a5357a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:07.332492 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b433f-ae47-4f08-9839-dbeec6a5357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:07.358710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-de1b433f-ae47-4f08-9839-dbeec6a5357a None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:16:07.360326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-de1b433f-ae47-4f08-9839-dbeec6a5357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:07.455128 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1423/2794] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:16:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:16:07.621188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:16:07.623519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:07.725750 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba490-799a-4e41-8c54-31c0e83c4fcc admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:16:07.731734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13bf58-624f-4a6b-b39a-1cd1ee4e4ceb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:16:07.733010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13bf58-624f-4a6b-b39a-1cd1ee4e4ceb service swift] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:07.733371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13bf58-624f-4a6b-b39a-1cd1ee4e4ceb service swift] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:07.733704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13bf58-624f-4a6b-b39a-1cd1ee4e4ceb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:07.743844 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13bf58-624f-4a6b-b39a-1cd1ee4e4ceb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:16:07.745663 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13bf58-624f-4a6b-b39a-1cd1ee4e4ceb service swift] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:16:07.755665 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1372/2795] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:16:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 12:16:17.445835 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:DELETE=5,keystone:SELECT=39,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:16:17.751460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:16:26.923017 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71b69-53da-4d5c-a325-f967e8c4352a None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:26.923657 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71b69-53da-4d5c-a325-f967e8c4352a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:26.924127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71b69-53da-4d5c-a325-f967e8c4352a None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:26.927062 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1424/2796] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:16:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:16:26.934928 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b355e-39a0-4175-995d-b13d2c8d8bc4 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:26.935815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b355e-39a0-4175-995d-b13d2c8d8bc4 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:26.936074 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b355e-39a0-4175-995d-b13d2c8d8bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:26.970150 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-596b355e-39a0-4175-995d-b13d2c8d8bc4 None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:16:26.972311 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-596b355e-39a0-4175-995d-b13d2c8d8bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:27.073905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1373/2797] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:16:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:16:27.087529 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71b69-53da-4d5c-a325-f967e8c4352a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:16:27.089883 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-64e71b69-53da-4d5c-a325-f967e8c4352a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:27.207905 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71b69-53da-4d5c-a325-f967e8c4352a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:16:27.211865 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff93a9-142d-4ee9-b595-490ceda0e28f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454fb811f6e34e1d881d4948ede058b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:16:27.212571 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff93a9-142d-4ee9-b595-490ceda0e28f service glance] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:27.212857 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff93a9-142d-4ee9-b595-490ceda0e28f service glance] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:27.213174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff93a9-142d-4ee9-b595-490ceda0e28f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:27.226025 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff93a9-142d-4ee9-b595-490ceda0e28f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:16:27.227645 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff93a9-142d-4ee9-b595-490ceda0e28f service glance] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:16:27.241506 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1425/2798] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:16:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:16:27.468302 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1dd9-cec9-4458-9bbf-7d720a576189 None None] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:27.468759 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1dd9-cec9-4458-9bbf-7d720a576189 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:27.469143 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1dd9-cec9-4458-9bbf-7d720a576189 None None] PATH_INFO: `/` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:27.470620 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1374/2799] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:16:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:16:27.476499 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c77b-b40f-422f-83b8-d51bf905b0d4 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:27.476889 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c77b-b40f-422f-83b8-d51bf905b0d4 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:27.483804 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c77b-b40f-422f-83b8-d51bf905b0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:27.522716 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-30a2c77b-b40f-422f-83b8-d51bf905b0d4 None None] MFA Rules not processed for user `073e43c049224af29e9f6d426ea47e84`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:16:27.524061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-30a2c77b-b40f-422f-83b8-d51bf905b0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:27.591486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1426/2800] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:16:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:16:27.606769 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1dd9-cec9-4458-9bbf-7d720a576189 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:16:27.608811 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-345e1dd9-cec9-4458-9bbf-7d720a576189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:27.623497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1dd9-cec9-4458-9bbf-7d720a576189 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:16:27.627628 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9267fb-c3b8-4486-a169-a5faab9fe4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:16:27.628345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9267fb-c3b8-4486-a169-a5faab9fe4cb service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:27.628655 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9267fb-c3b8-4486-a169-a5faab9fe4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:27.628968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9267fb-c3b8-4486-a169-a5faab9fe4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:27.638217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9267fb-c3b8-4486-a169-a5faab9fe4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:16:27.639704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9267fb-c3b8-4486-a169-a5faab9fe4cb service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:16:27.649380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1375/2801] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:16:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:16:27.754362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9f2b1-32d6-4ba4-b142-79f7cb087c03 None None] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:27.754699 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9f2b1-32d6-4ba4-b142-79f7cb087c03 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:27.755081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9f2b1-32d6-4ba4-b142-79f7cb087c03 None None] PATH_INFO: `/` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:27.756663 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1427/2802] 10.222.0.15 () {54 vars in 921 bytes} [Tue Oct 4 12:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:16:27.763541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9ffc6-e20f-40e7-a883-e8727c9b339e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:27.763943 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9ffc6-e20f-40e7-a883-e8727c9b339e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:27.764337 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9ffc6-e20f-40e7-a883-e8727c9b339e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:27.788439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-42f9ffc6-e20f-40e7-a883-e8727c9b339e None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:16:27.789762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-42f9ffc6-e20f-40e7-a883-e8727c9b339e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:27.801699 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1376/2803] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Oct 4 12:16:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:16:27.815739 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9f2b1-32d6-4ba4-b142-79f7cb087c03 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:16:27.830096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9f2b1-32d6-4ba4-b142-79f7cb087c03 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:16:27.834980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6703b5e-ba79-49bf-8378-a409492ae119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:16:27.835758 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6703b5e-ba79-49bf-8378-a409492ae119 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:27.836090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6703b5e-ba79-49bf-8378-a409492ae119 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:27.836464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6703b5e-ba79-49bf-8378-a409492ae119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:27.849819 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6703b5e-ba79-49bf-8378-a409492ae119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:16:27.851412 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6703b5e-ba79-49bf-8378-a409492ae119 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:16:27.864287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1428/2804] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:16:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:16:30.586993 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c20a0a-242d-4b65-a129-1ba718b18d71 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:30.587734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c20a0a-242d-4b65-a129-1ba718b18d71 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:30.587734 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c20a0a-242d-4b65-a129-1ba718b18d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:30.612348 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-b6c20a0a-242d-4b65-a129-1ba718b18d71 None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:16:30.613991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-b6c20a0a-242d-4b65-a129-1ba718b18d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:30.625503 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1377/2805] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:16:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:16:30.643549 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6703b5e-ba79-49bf-8378-a409492ae119 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:16:30.662826 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6703b5e-ba79-49bf-8378-a409492ae119 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:16:30.672400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9e47d-5b7f-48d3-be8e-09bb3ea856a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:16:30.673367 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9e47d-5b7f-48d3-be8e-09bb3ea856a1 service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:30.674375 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9e47d-5b7f-48d3-be8e-09bb3ea856a1 service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:30.674375 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9e47d-5b7f-48d3-be8e-09bb3ea856a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:30.687980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9e47d-5b7f-48d3-be8e-09bb3ea856a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:16:30.692797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9e47d-5b7f-48d3-be8e-09bb3ea856a1 service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:16:30.704859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1429/2806] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:16:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:16:40.603278 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=43 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:16:40.699514 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=76 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:16:46.265597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f157c9-41b4-4d22-a8d6-9f2193ed044b None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:46.266324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f157c9-41b4-4d22-a8d6-9f2193ed044b None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:46.266467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f157c9-41b4-4d22-a8d6-9f2193ed044b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:46.288119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-09f157c9-41b4-4d22-a8d6-9f2193ed044b None None] MFA Rules not processed for user `952ec01f4a174b6d94c3afd98f90ba81`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:16:46.289373 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-09f157c9-41b4-4d22-a8d6-9f2193ed044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:16:46.296743 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1378/2807] 10.222.0.15 () {58 vars in 967 bytes} [Tue Oct 4 12:16:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:16:46.311183 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9e47d-5b7f-48d3-be8e-09bb3ea856a1 service swift] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:16:46.325339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9e47d-5b7f-48d3-be8e-09bb3ea856a1 service swift] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:16:46.332274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6efc9-9cb0-4c3a-ac66-48a1c48a618a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734d8f03ff2949959cbfce98212a3ac6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:16:46.332947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6efc9-9cb0-4c3a-ac66-48a1c48a618a service swift] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:16:46.333342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6efc9-9cb0-4c3a-ac66-48a1c48a618a service swift] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:16:46.333741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6efc9-9cb0-4c3a-ac66-48a1c48a618a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:16:46.343801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6efc9-9cb0-4c3a-ac66-48a1c48a618a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:16:46.345477 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6efc9-9cb0-4c3a-ac66-48a1c48a618a service swift] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:16:46.354847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1430/2808] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Oct 4 12:16:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:16:56.282607 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:16:56.351186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=6 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:31.922431 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f157c9-41b4-4d22-a8d6-9f2193ed044b None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:31.924117 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-09f157c9-41b4-4d22-a8d6-9f2193ed044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:31.937571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f157c9-41b4-4d22-a8d6-9f2193ed044b None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:31.941482 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95d264-bbaa-4bde-a9da-0e33d8b50843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:31.942118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95d264-bbaa-4bde-a9da-0e33d8b50843 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:31.942380 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95d264-bbaa-4bde-a9da-0e33d8b50843 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:31.942648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95d264-bbaa-4bde-a9da-0e33d8b50843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:31.944643 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-2c95d264-bbaa-4bde-a9da-0e33d8b50843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:32.044371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95d264-bbaa-4bde-a9da-0e33d8b50843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:32.046290 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95d264-bbaa-4bde-a9da-0e33d8b50843 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:32.063637 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1379/2809] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:17:36.505550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6efc9-9cb0-4c3a-ac66-48a1c48a618a service swift] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:36.508232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-a9f6efc9-9cb0-4c3a-ac66-48a1c48a618a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:36.531863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6efc9-9cb0-4c3a-ac66-48a1c48a618a service swift] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:36.550404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:36.550404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:36.550404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:36.550404 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] PATH_INFO: `/v3/users/9ab97a61223f4719add7b3c0f845a319` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:36.551126 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab97a61223f4719add7b3c0f845a319)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:36.551126 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:36.765271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab97a61223f4719add7b3c0f845a319'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:36.766014 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab97a61223f4719add7b3c0f845a319'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:36.774142 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab97a61223f4719add7b3c0f845a319'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:36.774885 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab97a61223f4719add7b3c0f845a319'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:36.836701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-4e0ad68b-fc27-4b1b-a884-6488a0a3274b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab97a61223f4719add7b3c0f845a319'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:36.859074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1431/2810] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:17:36 2022] DELETE /identity/v3/users/9ab97a61223f4719add7b3c0f845a319 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:37.183497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95d264-bbaa-4bde-a9da-0e33d8b50843 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:37.205812 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95d264-bbaa-4bde-a9da-0e33d8b50843 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:37.292362 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:37.293313 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:37.293600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:37.293878 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] PATH_INFO: `/v3/projects/e41d4317234c458daad66f591d27c6fa` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:37.295213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e41d4317234c458daad66f591d27c6fa)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:37.297384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:37.459019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41d4317234c458daad66f591d27c6fa', 'request_id': 'req-21ac36a7-afad-49b1-9c80-b264842426aa'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:37.468091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41d4317234c458daad66f591d27c6fa', 'request_id': 'req-21ac36a7-afad-49b1-9c80-b264842426aa'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:37.481818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41d4317234c458daad66f591d27c6fa', 'request_id': 'req-21ac36a7-afad-49b1-9c80-b264842426aa'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:37.569439 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] The token cache is being invalidate because project e41d4317234c458daad66f591d27c6fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:17:37.569725 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:37.570531 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-21ac36a7-afad-49b1-9c80-b264842426aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:37.573103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1380/2811] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:17:37 2022] DELETE /identity/v3/projects/e41d4317234c458daad66f591d27c6fa => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:37.579582 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce7b5-7699-4884-bc7a-a099586563d5 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:37.580225 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce7b5-7699-4884-bc7a-a099586563d5 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:37.580713 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce7b5-7699-4884-bc7a-a099586563d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:37.607572 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-b8dce7b5-7699-4884-bc7a-a099586563d5 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:37.608936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-b8dce7b5-7699-4884-bc7a-a099586563d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:37.680314 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1432/2812] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:17:37.689776 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fc62b-2b79-48a9-ad60-743a060fe84c None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:37.690118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fc62b-2b79-48a9-ad60-743a060fe84c None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:37.690483 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fc62b-2b79-48a9-ad60-743a060fe84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:37.723329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f45fc62b-2b79-48a9-ad60-743a060fe84c None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:37.725455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f45fc62b-2b79-48a9-ad60-743a060fe84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:37.736893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1381/2813] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:17:37.741839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dce7b5-7699-4884-bc7a-a099586563d5 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:37.754248 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dce7b5-7699-4884-bc7a-a099586563d5 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:37.759346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d629f-d7d3-43de-b9bf-8ab595ddcab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:37.760089 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d629f-d7d3-43de-b9bf-8ab595ddcab9 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:37.760389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d629f-d7d3-43de-b9bf-8ab595ddcab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:37.760727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d629f-d7d3-43de-b9bf-8ab595ddcab9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:37.761409 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d629f-d7d3-43de-b9bf-8ab595ddcab9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:17:37.761718 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d629f-d7d3-43de-b9bf-8ab595ddcab9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:37.763368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d629f-d7d3-43de-b9bf-8ab595ddcab9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:37.777673 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1433/2814] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:17:37.782994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fc62b-2b79-48a9-ad60-743a060fe84c None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:37.794444 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fc62b-2b79-48a9-ad60-743a060fe84c None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:37.800034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd67df-d370-4dd9-87b7-998b1de6ee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:37.800706 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd67df-d370-4dd9-87b7-998b1de6ee1f admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:37.800985 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd67df-d370-4dd9-87b7-998b1de6ee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:37.801258 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd67df-d370-4dd9-87b7-998b1de6ee1f admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:37.802547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd67df-d370-4dd9-87b7-998b1de6ee1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:37.803957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd67df-d370-4dd9-87b7-998b1de6ee1f admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:37.823834 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:37.925235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1382/2815] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:17:37 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:17:37.930410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d629f-d7d3-43de-b9bf-8ab595ddcab9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:37.942894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d629f-d7d3-43de-b9bf-8ab595ddcab9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:38.020262 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761854f1-1a71-4bf1-8215-ad39c6028a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:38.020904 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761854f1-1a71-4bf1-8215-ad39c6028a59 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:38.021186 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761854f1-1a71-4bf1-8215-ad39c6028a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:38.021754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761854f1-1a71-4bf1-8215-ad39c6028a59 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:38.022447 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761854f1-1a71-4bf1-8215-ad39c6028a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:38.024638 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761854f1-1a71-4bf1-8215-ad39c6028a59 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:38.033897 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:38.108039 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1434/2816] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:17:37 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:17:38.112368 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd67df-d370-4dd9-87b7-998b1de6ee1f admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:38.125944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd67df-d370-4dd9-87b7-998b1de6ee1f admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:38.131275 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ede7e2-df2b-40d2-a998-ce927c765a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:38.132051 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ede7e2-df2b-40d2-a998-ce927c765a9c admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:38.132368 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ede7e2-df2b-40d2-a998-ce927c765a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:38.132644 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ede7e2-df2b-40d2-a998-ce927c765a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:38.133338 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ede7e2-df2b-40d2-a998-ce927c765a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:38.135900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ede7e2-df2b-40d2-a998-ce927c765a9c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:38.148420 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1383/2817] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:17:38.155288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761854f1-1a71-4bf1-8215-ad39c6028a59 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:38.167333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761854f1-1a71-4bf1-8215-ad39c6028a59 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:38.172586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6989cd-0e40-411b-9be9-c8e49430c957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:38.173229 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6989cd-0e40-411b-9be9-c8e49430c957 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:38.173511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6989cd-0e40-411b-9be9-c8e49430c957 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:38.173786 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6989cd-0e40-411b-9be9-c8e49430c957 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:38.174461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6989cd-0e40-411b-9be9-c8e49430c957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:38.176151 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6989cd-0e40-411b-9be9-c8e49430c957 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:38.189894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1435/2818] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:17:38.195359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ede7e2-df2b-40d2-a998-ce927c765a9c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:38.207498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ede7e2-df2b-40d2-a998-ce927c765a9c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:38.212889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03313886-89db-427a-a50a-e16d41159758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:38.213864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03313886-89db-427a-a50a-e16d41159758 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:38.214195 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03313886-89db-427a-a50a-e16d41159758 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:38.214520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03313886-89db-427a-a50a-e16d41159758 admin admin] PATH_INFO: `/v3/projects/26b44051c3ed4ccaac86fd204899886b/users/2c6b15d2fb484260876a3401a9e2dc87/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:38.233400 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03313886-89db-427a-a50a-e16d41159758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b44051c3ed4ccaac86fd204899886b, user_id=2c6b15d2fb484260876a3401a9e2dc87, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:38.236342 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03313886-89db-427a-a50a-e16d41159758 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:38.305781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1384/2819] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:17:38 2022] PUT /identity/v3/projects/26b44051c3ed4ccaac86fd204899886b/users/2c6b15d2fb484260876a3401a9e2dc87/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:38.317682 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8efb9-d140-42e0-96dd-b4157dc2c666 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:38.317997 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8efb9-d140-42e0-96dd-b4157dc2c666 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:38.318342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8efb9-d140-42e0-96dd-b4157dc2c666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:38.352481 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-77a8efb9-d140-42e0-96dd-b4157dc2c666 None None] MFA Rules not processed for user `2c6b15d2fb484260876a3401a9e2dc87`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:38.354307 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-77a8efb9-d140-42e0-96dd-b4157dc2c666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:38.435265 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1436/2820] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:17:38.448157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03313886-89db-427a-a50a-e16d41159758 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:38.450102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-03313886-89db-427a-a50a-e16d41159758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:38.545625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03313886-89db-427a-a50a-e16d41159758 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:38.552086 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea173-bde8-41ee-b9f0-d64aafb9289d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed14c1bc105d4c5cade5813edc1f0dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:38.552881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea173-bde8-41ee-b9f0-d64aafb9289d service nova] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:38.553235 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea173-bde8-41ee-b9f0-d64aafb9289d service nova] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:38.553612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea173-bde8-41ee-b9f0-d64aafb9289d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:38.563209 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ea173-bde8-41ee-b9f0-d64aafb9289d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:38.565220 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ea173-bde8-41ee-b9f0-d64aafb9289d service nova] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:38.576883 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1385/2821] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:17:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:17:38.696490 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8efb9-d140-42e0-96dd-b4157dc2c666 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:38.699011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-77a8efb9-d140-42e0-96dd-b4157dc2c666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:38.770984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8efb9-d140-42e0-96dd-b4157dc2c666 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:38.776154 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e2ef-8aba-4802-b669-41f9a4542208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:38.777075 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e2ef-8aba-4802-b669-41f9a4542208 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:38.777446 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e2ef-8aba-4802-b669-41f9a4542208 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:38.777868 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e2ef-8aba-4802-b669-41f9a4542208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:38.779832 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-7bf2e2ef-8aba-4802-b669-41f9a4542208 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:38.799368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e2ef-8aba-4802-b669-41f9a4542208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:38.801896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e2ef-8aba-4802-b669-41f9a4542208 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:38.818211 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1437/2822] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:17:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:17:39.019539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea173-bde8-41ee-b9f0-d64aafb9289d service nova] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:39.100005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea173-bde8-41ee-b9f0-d64aafb9289d service nova] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:39.105295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:39.106105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:39.106429 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:39.106737 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] PATH_INFO: `/v3/users/2c6b15d2fb484260876a3401a9e2dc87` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:39.108032 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6b15d2fb484260876a3401a9e2dc87)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:39.110273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:39.447808 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6b15d2fb484260876a3401a9e2dc87'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:39.449211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:39.457042 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6b15d2fb484260876a3401a9e2dc87'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:39.457932 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6b15d2fb484260876a3401a9e2dc87'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:39.613408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6b15d2fb484260876a3401a9e2dc87'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:39.629843 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6b15d2fb484260876a3401a9e2dc87'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:39.636931 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1386/2823] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:17:39 2022] DELETE /identity/v3/users/2c6b15d2fb484260876a3401a9e2dc87 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:39.642121 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d084db-acc0-43b6-85cc-a389a5aba57d None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:39.642405 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d084db-acc0-43b6-85cc-a389a5aba57d None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:39.642680 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d084db-acc0-43b6-85cc-a389a5aba57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:39.665362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-47d084db-acc0-43b6-85cc-a389a5aba57d None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:39.667558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-47d084db-acc0-43b6-85cc-a389a5aba57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:39.738551 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1438/2824] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:17:39.749770 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:39.751261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:39.829157 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b2e45-8bdc-4eef-9aa6-9f43eb65bffb admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:39.832994 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e944fd-3010-42a2-91db-54e41324a3a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:39.833612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e944fd-3010-42a2-91db-54e41324a3a2 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:39.833892 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e944fd-3010-42a2-91db-54e41324a3a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:39.834165 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e944fd-3010-42a2-91db-54e41324a3a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:39.846020 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e944fd-3010-42a2-91db-54e41324a3a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:39.855871 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e944fd-3010-42a2-91db-54e41324a3a2 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:39.858978 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1387/2825] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:17:40.388297 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d084db-acc0-43b6-85cc-a389a5aba57d None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:40.390411 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-47d084db-acc0-43b6-85cc-a389a5aba57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:40.412381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d084db-acc0-43b6-85cc-a389a5aba57d None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:40.421021 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:40.422011 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:40.422195 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:40.422475 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] PATH_INFO: `/v3/projects/26b44051c3ed4ccaac86fd204899886b` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:40.424198 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b44051c3ed4ccaac86fd204899886b)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:40.426787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:40.436143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:40.588153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b44051c3ed4ccaac86fd204899886b', 'request_id': 'req-29bcc9ff-444f-491f-91d3-77770df9baf7'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:40.594690 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b44051c3ed4ccaac86fd204899886b', 'request_id': 'req-29bcc9ff-444f-491f-91d3-77770df9baf7'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:40.687476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b44051c3ed4ccaac86fd204899886b', 'request_id': 'req-29bcc9ff-444f-491f-91d3-77770df9baf7'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:40.702131 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] The token cache is being invalidate because project 26b44051c3ed4ccaac86fd204899886b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:17:40.702459 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:40.703180 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-29bcc9ff-444f-491f-91d3-77770df9baf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:40.705053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1439/2826] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:17:40 2022] DELETE /identity/v3/projects/26b44051c3ed4ccaac86fd204899886b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:40.710611 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb595a5-0d8e-4f2a-a1b5-d32f83c721af None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:40.711467 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb595a5-0d8e-4f2a-a1b5-d32f83c721af None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:40.711679 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb595a5-0d8e-4f2a-a1b5-d32f83c721af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:40.735123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7eb595a5-0d8e-4f2a-a1b5-d32f83c721af None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:40.736977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7eb595a5-0d8e-4f2a-a1b5-d32f83c721af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:40.816030 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1388/2827] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:17:40.885116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec11173-99f4-4679-ae07-b283296c7688 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:40.886034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec11173-99f4-4679-ae07-b283296c7688 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:40.886034 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec11173-99f4-4679-ae07-b283296c7688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:40.908014 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9ec11173-99f4-4679-ae07-b283296c7688 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:40.909389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9ec11173-99f4-4679-ae07-b283296c7688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:40.919394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1440/2828] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:17:40.924125 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb595a5-0d8e-4f2a-a1b5-d32f83c721af None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:40.936072 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb595a5-0d8e-4f2a-a1b5-d32f83c721af None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:40.941139 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586997b2-2aa7-44a2-8488-9007638bafc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:40.941681 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586997b2-2aa7-44a2-8488-9007638bafc6 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:40.941958 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586997b2-2aa7-44a2-8488-9007638bafc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:40.942227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586997b2-2aa7-44a2-8488-9007638bafc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:40.942935 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586997b2-2aa7-44a2-8488-9007638bafc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:17:40.943240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586997b2-2aa7-44a2-8488-9007638bafc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:40.944957 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586997b2-2aa7-44a2-8488-9007638bafc6 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:40.959102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1389/2829] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:17:40.964433 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec11173-99f4-4679-ae07-b283296c7688 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:40.974581 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec11173-99f4-4679-ae07-b283296c7688 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:40.979809 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce35a1-bfcd-45bc-9fe4-b4fb2fb6d158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:40.980521 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce35a1-bfcd-45bc-9fe4-b4fb2fb6d158 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:40.980803 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce35a1-bfcd-45bc-9fe4-b4fb2fb6d158 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:40.981081 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce35a1-bfcd-45bc-9fe4-b4fb2fb6d158 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:40.981748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce35a1-bfcd-45bc-9fe4-b4fb2fb6d158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:40.983731 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce35a1-bfcd-45bc-9fe4-b4fb2fb6d158 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:41.087348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1441/2830] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:17:40 2022] POST /identity/v3/projects => generated 405 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:17:41.091837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586997b2-2aa7-44a2-8488-9007638bafc6 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:41.102872 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586997b2-2aa7-44a2-8488-9007638bafc6 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:41.170192 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:41.177822 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4039f-c2f7-42b0-9f57-dd2b8530da68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:41.178437 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4039f-c2f7-42b0-9f57-dd2b8530da68 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:41.178716 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4039f-c2f7-42b0-9f57-dd2b8530da68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:41.179120 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4039f-c2f7-42b0-9f57-dd2b8530da68 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:41.179736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4039f-c2f7-42b0-9f57-dd2b8530da68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:41.182413 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4039f-c2f7-42b0-9f57-dd2b8530da68 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:41.295019 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1390/2831] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:17:41 2022] POST /identity/v3/users => generated 303 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:17:41.300381 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce35a1-bfcd-45bc-9fe4-b4fb2fb6d158 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:41.312472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce35a1-bfcd-45bc-9fe4-b4fb2fb6d158 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:41.317777 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7be02e-1cee-4412-8cbe-3bb8962a0482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:41.318438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7be02e-1cee-4412-8cbe-3bb8962a0482 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:41.318706 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7be02e-1cee-4412-8cbe-3bb8962a0482 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:41.318970 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7be02e-1cee-4412-8cbe-3bb8962a0482 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:41.319632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7be02e-1cee-4412-8cbe-3bb8962a0482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:41.321450 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7be02e-1cee-4412-8cbe-3bb8962a0482 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:41.333867 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1442/2832] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:17:41.343018 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4039f-c2f7-42b0-9f57-dd2b8530da68 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:41.354781 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4039f-c2f7-42b0-9f57-dd2b8530da68 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:41.360100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cad51-fc62-49ff-81e6-19f1664b27f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:41.360766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cad51-fc62-49ff-81e6-19f1664b27f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:41.361394 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cad51-fc62-49ff-81e6-19f1664b27f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:41.362010 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cad51-fc62-49ff-81e6-19f1664b27f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:41.363696 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cad51-fc62-49ff-81e6-19f1664b27f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:41.366588 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cad51-fc62-49ff-81e6-19f1664b27f2 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:41.381187 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1391/2833] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:17:41.385914 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7be02e-1cee-4412-8cbe-3bb8962a0482 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:41.397109 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7be02e-1cee-4412-8cbe-3bb8962a0482 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:41.403122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21262c-4671-4348-9ce5-d0eb32cc93f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:41.403870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21262c-4671-4348-9ce5-d0eb32cc93f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:41.404252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21262c-4671-4348-9ce5-d0eb32cc93f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:41.404666 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21262c-4671-4348-9ce5-d0eb32cc93f9 admin admin] PATH_INFO: `/v3/projects/d3f82f3fc81440938c4029fc14796138/users/a5cd3c87054543ef86fdf56e9d554f41/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:41.422722 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21262c-4671-4348-9ce5-d0eb32cc93f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f82f3fc81440938c4029fc14796138, user_id=a5cd3c87054543ef86fdf56e9d554f41, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:41.425575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21262c-4671-4348-9ce5-d0eb32cc93f9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:41.512333 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1443/2834] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:17:41 2022] PUT /identity/v3/projects/d3f82f3fc81440938c4029fc14796138/users/a5cd3c87054543ef86fdf56e9d554f41/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:41.517981 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934c03-ef2d-4d7b-90c1-c1e994f32502 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:41.518269 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934c03-ef2d-4d7b-90c1-c1e994f32502 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:41.518558 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934c03-ef2d-4d7b-90c1-c1e994f32502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:41.541325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-d7934c03-ef2d-4d7b-90c1-c1e994f32502 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:41.542592 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-d7934c03-ef2d-4d7b-90c1-c1e994f32502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:41.623721 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1392/2835] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:17:41.637400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21262c-4671-4348-9ce5-d0eb32cc93f9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:41.637400 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-8b21262c-4671-4348-9ce5-d0eb32cc93f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:41.703274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21262c-4671-4348-9ce5-d0eb32cc93f9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:41.711546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbff351-13d3-49c2-bd48-206cdea7c3a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:41.712998 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbff351-13d3-49c2-bd48-206cdea7c3a5 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:41.713413 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbff351-13d3-49c2-bd48-206cdea7c3a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:41.713822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbff351-13d3-49c2-bd48-206cdea7c3a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:41.729218 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbff351-13d3-49c2-bd48-206cdea7c3a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:41.731680 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbff351-13d3-49c2-bd48-206cdea7c3a5 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:41.749749 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1444/2836] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:17:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:17:43.428479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc02a9c-3ede-43a8-9a31-6d8e93bd7956 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:43.429326 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc02a9c-3ede-43a8-9a31-6d8e93bd7956 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:43.429567 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc02a9c-3ede-43a8-9a31-6d8e93bd7956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:43.476603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-5fc02a9c-3ede-43a8-9a31-6d8e93bd7956 None None] MFA Rules not processed for user `a5cd3c87054543ef86fdf56e9d554f41`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:43.479137 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5fc02a9c-3ede-43a8-9a31-6d8e93bd7956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:43.590584 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1393/2837] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:43 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:17:43.596023 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbff351-13d3-49c2-bd48-206cdea7c3a5 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:43.615291 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbff351-13d3-49c2-bd48-206cdea7c3a5 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:43.623849 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11788c73-38f5-4177-9d39-a3b8fafd102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:43.624956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11788c73-38f5-4177-9d39-a3b8fafd102c admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:43.625468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11788c73-38f5-4177-9d39-a3b8fafd102c admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:43.625968 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11788c73-38f5-4177-9d39-a3b8fafd102c admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:43.627214 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11788c73-38f5-4177-9d39-a3b8fafd102c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:43.631272 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11788c73-38f5-4177-9d39-a3b8fafd102c admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:43.776587 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1445/2838] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:17:43 2022] POST /identity/v3/projects => generated 401 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:17:43.781205 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc02a9c-3ede-43a8-9a31-6d8e93bd7956 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:43.794972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc02a9c-3ede-43a8-9a31-6d8e93bd7956 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:43.870016 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6fa0f-5d5d-411c-b98f-9bf5e34038fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:43.870796 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6fa0f-5d5d-411c-b98f-9bf5e34038fc admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:43.871188 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6fa0f-5d5d-411c-b98f-9bf5e34038fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:43.871559 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6fa0f-5d5d-411c-b98f-9bf5e34038fc admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:43.872365 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6fa0f-5d5d-411c-b98f-9bf5e34038fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:43.874466 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6fa0f-5d5d-411c-b98f-9bf5e34038fc admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:43.886028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:43.997670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1394/2839] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:17:43 2022] POST /identity/v3/users => generated 307 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:17:44.003188 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11788c73-38f5-4177-9d39-a3b8fafd102c admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:44.023161 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11788c73-38f5-4177-9d39-a3b8fafd102c admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:44.030894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09099168-747e-4ebf-9d04-2e8f04ebb477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:44.031890 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099168-747e-4ebf-9d04-2e8f04ebb477 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:44.032421 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099168-747e-4ebf-9d04-2e8f04ebb477 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:44.032859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09099168-747e-4ebf-9d04-2e8f04ebb477 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:44.034035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09099168-747e-4ebf-9d04-2e8f04ebb477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:44.037061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09099168-747e-4ebf-9d04-2e8f04ebb477 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:44.049891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:44.056648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1446/2840] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:17:44.061345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6fa0f-5d5d-411c-b98f-9bf5e34038fc admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:44.072649 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6fa0f-5d5d-411c-b98f-9bf5e34038fc admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:44.077752 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab92ba-b538-4c0b-aa1c-43c1309568ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:44.078435 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab92ba-b538-4c0b-aa1c-43c1309568ca admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:44.078714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab92ba-b538-4c0b-aa1c-43c1309568ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:44.078990 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab92ba-b538-4c0b-aa1c-43c1309568ca admin admin] PATH_INFO: `/v3/projects/5b97b31cab2a45e295e25fe84b82afc1/users/4d5b44b4be1a41aba70326d5536d9989/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:44.097118 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab92ba-b538-4c0b-aa1c-43c1309568ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b97b31cab2a45e295e25fe84b82afc1, user_id=4d5b44b4be1a41aba70326d5536d9989, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:44.101084 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab92ba-b538-4c0b-aa1c-43c1309568ca admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:44.186780 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1395/2841] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:17:44 2022] PUT /identity/v3/projects/5b97b31cab2a45e295e25fe84b82afc1/users/4d5b44b4be1a41aba70326d5536d9989/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:44.191251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09099168-747e-4ebf-9d04-2e8f04ebb477 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:44.203366 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09099168-747e-4ebf-9d04-2e8f04ebb477 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:44.282207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25f8b8-4270-4144-b49f-a75025aa0de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:44.282851 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25f8b8-4270-4144-b49f-a75025aa0de2 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:44.283137 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25f8b8-4270-4144-b49f-a75025aa0de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:44.283418 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25f8b8-4270-4144-b49f-a75025aa0de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:44.284078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25f8b8-4270-4144-b49f-a75025aa0de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:44.285912 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25f8b8-4270-4144-b49f-a75025aa0de2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:44.298419 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1447/2842] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:17:44.302930 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab92ba-b538-4c0b-aa1c-43c1309568ca admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:44.317687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab92ba-b538-4c0b-aa1c-43c1309568ca admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:44.326257 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a0a35-8669-494f-b87a-7982be3fbbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:44.327432 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a0a35-8669-494f-b87a-7982be3fbbab admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:44.327898 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a0a35-8669-494f-b87a-7982be3fbbab admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:44.328650 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a0a35-8669-494f-b87a-7982be3fbbab admin admin] PATH_INFO: `/v3/projects/5b97b31cab2a45e295e25fe84b82afc1/users/4d5b44b4be1a41aba70326d5536d9989/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:44.332341 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a0a35-8669-494f-b87a-7982be3fbbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b97b31cab2a45e295e25fe84b82afc1, user_id=4d5b44b4be1a41aba70326d5536d9989, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:44.337603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a0a35-8669-494f-b87a-7982be3fbbab admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:44.356154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1396/2843] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:17:44 2022] PUT /identity/v3/projects/5b97b31cab2a45e295e25fe84b82afc1/users/4d5b44b4be1a41aba70326d5536d9989/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:45.927205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47158f2-afa5-435d-86a0-88983e6aa675 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:45.928099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47158f2-afa5-435d-86a0-88983e6aa675 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:45.928099 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47158f2-afa5-435d-86a0-88983e6aa675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:45.973480 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f47158f2-afa5-435d-86a0-88983e6aa675 None None] MFA Rules not processed for user `4d5b44b4be1a41aba70326d5536d9989`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:45.975038 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f47158f2-afa5-435d-86a0-88983e6aa675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:46.063911 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1448/2844] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:45 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:17:47.677331 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a0a35-8669-494f-b87a-7982be3fbbab admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:47.762045 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a0a35-8669-494f-b87a-7982be3fbbab admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:47.768270 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:47.768938 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:47.769219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:47.769495 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] PATH_INFO: `/v3/users/a5cd3c87054543ef86fdf56e9d554f41` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:47.770760 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5cd3c87054543ef86fdf56e9d554f41)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:47.772763 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:48.010088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5cd3c87054543ef86fdf56e9d554f41'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:48.019241 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5cd3c87054543ef86fdf56e9d554f41'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:48.019926 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5cd3c87054543ef86fdf56e9d554f41'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:48.098991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5cd3c87054543ef86fdf56e9d554f41'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:48.115092 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5cd3c87054543ef86fdf56e9d554f41'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:48.122657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1397/2845] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:17:47 2022] DELETE /identity/v3/users/a5cd3c87054543ef86fdf56e9d554f41 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:17:48.467634 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47158f2-afa5-435d-86a0-88983e6aa675 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:48.482423 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47158f2-afa5-435d-86a0-88983e6aa675 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:48.559694 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:48.560712 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:48.561244 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:48.561575 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] PATH_INFO: `/v3/projects/d3f82f3fc81440938c4029fc14796138` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:48.563199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f82f3fc81440938c4029fc14796138)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:48.565725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:48.596053 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:48.751701 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f82f3fc81440938c4029fc14796138', 'request_id': 'req-75a5560e-cc44-4e49-9294-678bfe141796'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:48.760651 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f82f3fc81440938c4029fc14796138', 'request_id': 'req-75a5560e-cc44-4e49-9294-678bfe141796'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:48.853476 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f82f3fc81440938c4029fc14796138', 'request_id': 'req-75a5560e-cc44-4e49-9294-678bfe141796'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:48.868152 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] The token cache is being invalidate because project d3f82f3fc81440938c4029fc14796138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:17:48.868571 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:48.869318 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:48.870942 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1449/2846] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:17:48 2022] DELETE /identity/v3/projects/d3f82f3fc81440938c4029fc14796138 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:48.875520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:48.877442 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:48.893639 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa9f14-a3ba-4ab0-8a1c-6677ae0e5d7c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:48.942033 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:48.974281 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:48.974963 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:48.975242 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:48.975515 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] PATH_INFO: `/v3/users/4d5b44b4be1a41aba70326d5536d9989` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:48.976748 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d5b44b4be1a41aba70326d5536d9989)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:48.978861 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:49.202196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d5b44b4be1a41aba70326d5536d9989'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:49.211529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d5b44b4be1a41aba70326d5536d9989'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:49.212438 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d5b44b4be1a41aba70326d5536d9989'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:49.309900 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d5b44b4be1a41aba70326d5536d9989'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:49.325587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-4d9a95d2-6927-458c-8259-c8c51c12dc8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d5b44b4be1a41aba70326d5536d9989'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:49.333082 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1398/2847] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:17:48 2022] DELETE /identity/v3/users/4d5b44b4be1a41aba70326d5536d9989 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:49.725535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:49.727339 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:49.745503 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5560e-cc44-4e49-9294-678bfe141796 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:49.834900 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:49.835710 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:49.836127 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:49.836550 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] PATH_INFO: `/v3/projects/5b97b31cab2a45e295e25fe84b82afc1` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:49.838479 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b97b31cab2a45e295e25fe84b82afc1)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:49.840864 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:49.998146 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b97b31cab2a45e295e25fe84b82afc1', 'request_id': 'req-662810a1-90e5-4a6f-9b8b-7b0aac8df862'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:50.006892 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b97b31cab2a45e295e25fe84b82afc1', 'request_id': 'req-662810a1-90e5-4a6f-9b8b-7b0aac8df862'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:50.105145 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b97b31cab2a45e295e25fe84b82afc1', 'request_id': 'req-662810a1-90e5-4a6f-9b8b-7b0aac8df862'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:50.118991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] The token cache is being invalidate because project 5b97b31cab2a45e295e25fe84b82afc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:17:50.119330 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:50.119977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-662810a1-90e5-4a6f-9b8b-7b0aac8df862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:17:50.121765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1450/2848] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:17:49 2022] DELETE /identity/v3/projects/5b97b31cab2a45e295e25fe84b82afc1 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:50.127955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898827da-ec42-4027-96eb-c8b915c460f2 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:50.128579 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898827da-ec42-4027-96eb-c8b915c460f2 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:50.128786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898827da-ec42-4027-96eb-c8b915c460f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:50.153378 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-898827da-ec42-4027-96eb-c8b915c460f2 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:50.155229 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-898827da-ec42-4027-96eb-c8b915c460f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:50.269258 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1399/2849] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:17:50.280070 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e632e4-57ee-4e2d-ab0c-fe457015c926 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:50.280397 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e632e4-57ee-4e2d-ab0c-fe457015c926 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:50.280680 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e632e4-57ee-4e2d-ab0c-fe457015c926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:50.303489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-70e632e4-57ee-4e2d-ab0c-fe457015c926 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:50.304801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-70e632e4-57ee-4e2d-ab0c-fe457015c926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:50.313527 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1451/2850] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:17:50.318319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898827da-ec42-4027-96eb-c8b915c460f2 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:50.330509 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898827da-ec42-4027-96eb-c8b915c460f2 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:50.335657 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6cec7-0c6e-4fb1-830b-9e5f57133d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:50.336344 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6cec7-0c6e-4fb1-830b-9e5f57133d31 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:50.336625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6cec7-0c6e-4fb1-830b-9e5f57133d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:50.337536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6cec7-0c6e-4fb1-830b-9e5f57133d31 admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:50.338222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6cec7-0c6e-4fb1-830b-9e5f57133d31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:17:50.338532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6cec7-0c6e-4fb1-830b-9e5f57133d31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:50.340212 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6cec7-0c6e-4fb1-830b-9e5f57133d31 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:50.353958 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1400/2851] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:17:50.360721 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e632e4-57ee-4e2d-ab0c-fe457015c926 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:50.379843 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e632e4-57ee-4e2d-ab0c-fe457015c926 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:50.388511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66d6a5-5bde-438a-8853-2d2647a7f3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:50.389620 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66d6a5-5bde-438a-8853-2d2647a7f3fb admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:50.390085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66d6a5-5bde-438a-8853-2d2647a7f3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:50.390577 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66d6a5-5bde-438a-8853-2d2647a7f3fb admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:50.391621 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66d6a5-5bde-438a-8853-2d2647a7f3fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:50.395347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66d6a5-5bde-438a-8853-2d2647a7f3fb admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:50.508632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1452/2852] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:17:50 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:17:50.514055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6cec7-0c6e-4fb1-830b-9e5f57133d31 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:50.535908 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6cec7-0c6e-4fb1-830b-9e5f57133d31 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:50.595610 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:50.615530 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bc7d6-80f2-4c03-a774-27123e8f1712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:50.616505 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bc7d6-80f2-4c03-a774-27123e8f1712 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:50.617690 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bc7d6-80f2-4c03-a774-27123e8f1712 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:50.618603 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bc7d6-80f2-4c03-a774-27123e8f1712 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:50.619683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bc7d6-80f2-4c03-a774-27123e8f1712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:50.621842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bc7d6-80f2-4c03-a774-27123e8f1712 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:50.744405 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1401/2853] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:17:50 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:17:50.750201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66d6a5-5bde-438a-8853-2d2647a7f3fb admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:50.761468 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66d6a5-5bde-438a-8853-2d2647a7f3fb admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:50.768344 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d02bcf-2ac2-424d-8131-67f5b25396a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:50.768907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02bcf-2ac2-424d-8131-67f5b25396a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:50.769190 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02bcf-2ac2-424d-8131-67f5b25396a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:50.769463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02bcf-2ac2-424d-8131-67f5b25396a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:50.770125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d02bcf-2ac2-424d-8131-67f5b25396a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:50.771776 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d02bcf-2ac2-424d-8131-67f5b25396a2 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:50.784104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1453/2854] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:17:50.789318 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bc7d6-80f2-4c03-a774-27123e8f1712 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:50.812161 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bc7d6-80f2-4c03-a774-27123e8f1712 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:50.818153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de77a8-c59a-405e-a397-b017487d5052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:50.818806 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de77a8-c59a-405e-a397-b017487d5052 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:50.819088 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de77a8-c59a-405e-a397-b017487d5052 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:50.819364 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de77a8-c59a-405e-a397-b017487d5052 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:50.820024 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de77a8-c59a-405e-a397-b017487d5052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:50.822312 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de77a8-c59a-405e-a397-b017487d5052 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:50.835911 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1402/2855] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:17:50.842258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d02bcf-2ac2-424d-8131-67f5b25396a2 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:50.848083 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:50.854332 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d02bcf-2ac2-424d-8131-67f5b25396a2 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:50.860043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c254c77-b9aa-4bbc-a1b6-6a449b4fbdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:50.860963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c254c77-b9aa-4bbc-a1b6-6a449b4fbdd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:50.861296 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c254c77-b9aa-4bbc-a1b6-6a449b4fbdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:50.861633 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c254c77-b9aa-4bbc-a1b6-6a449b4fbdd8 admin admin] PATH_INFO: `/v3/projects/daa12ae821ff4d50943f51b76140f2e1/users/b1efe00df1724424816f71332367c46d/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:50.880991 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c254c77-b9aa-4bbc-a1b6-6a449b4fbdd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa12ae821ff4d50943f51b76140f2e1, user_id=b1efe00df1724424816f71332367c46d, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:50.884437 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c254c77-b9aa-4bbc-a1b6-6a449b4fbdd8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:51.002295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1454/2856] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:17:50 2022] PUT /identity/v3/projects/daa12ae821ff4d50943f51b76140f2e1/users/b1efe00df1724424816f71332367c46d/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:51.015168 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2ea22-e6e8-4aa5-b320-ef67c72af3f5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:51.019672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2ea22-e6e8-4aa5-b320-ef67c72af3f5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:51.019672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2ea22-e6e8-4aa5-b320-ef67c72af3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:51.049265 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-18e2ea22-e6e8-4aa5-b320-ef67c72af3f5 None None] MFA Rules not processed for user `b1efe00df1724424816f71332367c46d`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:51.050564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-18e2ea22-e6e8-4aa5-b320-ef67c72af3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:51.128864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1403/2857] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:17:51.133937 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c254c77-b9aa-4bbc-a1b6-6a449b4fbdd8 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:51.208698 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c254c77-b9aa-4bbc-a1b6-6a449b4fbdd8 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:51.216185 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f2163-8f97-4220-9530-46ca6c6e3542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:51.217203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f2163-8f97-4220-9530-46ca6c6e3542 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:51.217609 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f2163-8f97-4220-9530-46ca6c6e3542 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:51.218091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f2163-8f97-4220-9530-46ca6c6e3542 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:51.219122 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f2163-8f97-4220-9530-46ca6c6e3542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:51.222353 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f2163-8f97-4220-9530-46ca6c6e3542 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:51.324074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1455/2858] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:17:51 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:17:51.328581 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2ea22-e6e8-4aa5-b320-ef67c72af3f5 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:51.341416 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2ea22-e6e8-4aa5-b320-ef67c72af3f5 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:51.411264 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc5e50-b751-4678-a89f-1f16289a3aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:51.412009 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc5e50-b751-4678-a89f-1f16289a3aff admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:51.412375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc5e50-b751-4678-a89f-1f16289a3aff admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:51.412704 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc5e50-b751-4678-a89f-1f16289a3aff admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:51.413402 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc5e50-b751-4678-a89f-1f16289a3aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:51.416286 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc5e50-b751-4678-a89f-1f16289a3aff admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:51.632254 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1404/2859] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:17:51 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:17:51.726158 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f2163-8f97-4220-9530-46ca6c6e3542 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:51.726158 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f2163-8f97-4220-9530-46ca6c6e3542 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:51.726158 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7093f-2309-47a1-bedf-26527279a8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:51.726158 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7093f-2309-47a1-bedf-26527279a8ad admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:51.726822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7093f-2309-47a1-bedf-26527279a8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:51.726822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7093f-2309-47a1-bedf-26527279a8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:51.726822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7093f-2309-47a1-bedf-26527279a8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:51.726822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7093f-2309-47a1-bedf-26527279a8ad admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:51.726822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1456/2860] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:17:51.728034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc5e50-b751-4678-a89f-1f16289a3aff admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:51.728034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc5e50-b751-4678-a89f-1f16289a3aff admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:51.728034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955a24f-0d0e-4c79-ba50-c29ac4b61c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:51.728034 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955a24f-0d0e-4c79-ba50-c29ac4b61c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:51.728625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955a24f-0d0e-4c79-ba50-c29ac4b61c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:51.728625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955a24f-0d0e-4c79-ba50-c29ac4b61c77 admin admin] PATH_INFO: `/v3/projects/bc287ba0857d45a0982c2f47024eaeef/users/0f9eca0c2aea476ab7d5e533aaf8a5ff/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:51.728625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955a24f-0d0e-4c79-ba50-c29ac4b61c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc287ba0857d45a0982c2f47024eaeef, user_id=0f9eca0c2aea476ab7d5e533aaf8a5ff, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:51.728625 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955a24f-0d0e-4c79-ba50-c29ac4b61c77 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:51.996507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1405/2861] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:17:51 2022] PUT /identity/v3/projects/bc287ba0857d45a0982c2f47024eaeef/users/0f9eca0c2aea476ab7d5e533aaf8a5ff/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:17:52.001006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7093f-2309-47a1-bedf-26527279a8ad admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:52.013576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7093f-2309-47a1-bedf-26527279a8ad admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:52.081848 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a04e5c-7352-47f1-ab10-32624cb587f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:52.082478 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a04e5c-7352-47f1-ab10-32624cb587f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:52.082763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a04e5c-7352-47f1-ab10-32624cb587f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:52.083041 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a04e5c-7352-47f1-ab10-32624cb587f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:52.083727 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a04e5c-7352-47f1-ab10-32624cb587f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:52.085407 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a04e5c-7352-47f1-ab10-32624cb587f5 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:52.097774 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1457/2862] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:17:52.105528 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955a24f-0d0e-4c79-ba50-c29ac4b61c77 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:52.118851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955a24f-0d0e-4c79-ba50-c29ac4b61c77 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:52.123953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f39e39-0dc9-4865-aa5e-36b1fad673f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:52.124678 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39e39-0dc9-4865-aa5e-36b1fad673f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:52.124959 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39e39-0dc9-4865-aa5e-36b1fad673f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:52.125234 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39e39-0dc9-4865-aa5e-36b1fad673f9 admin admin] PATH_INFO: `/v3/projects/bc287ba0857d45a0982c2f47024eaeef/users/0f9eca0c2aea476ab7d5e533aaf8a5ff/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:52.127288 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f39e39-0dc9-4865-aa5e-36b1fad673f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc287ba0857d45a0982c2f47024eaeef, user_id=0f9eca0c2aea476ab7d5e533aaf8a5ff, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:52.130295 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f39e39-0dc9-4865-aa5e-36b1fad673f9 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:52.143213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1406/2863] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:17:52 2022] PUT /identity/v3/projects/bc287ba0857d45a0982c2f47024eaeef/users/0f9eca0c2aea476ab7d5e533aaf8a5ff/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:17:52.155535 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266609b8-e885-435c-ad7b-1c1a289a03e7 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:52.155859 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266609b8-e885-435c-ad7b-1c1a289a03e7 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:52.156085 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266609b8-e885-435c-ad7b-1c1a289a03e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:52.188616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-266609b8-e885-435c-ad7b-1c1a289a03e7 None None] MFA Rules not processed for user `0f9eca0c2aea476ab7d5e533aaf8a5ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:17:52.190374 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-266609b8-e885-435c-ad7b-1c1a289a03e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:52.237903 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:52.275742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1458/2864] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:17:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:17:52.288126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f39e39-0dc9-4865-aa5e-36b1fad673f9 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:52.290253 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-e6f39e39-0dc9-4865-aa5e-36b1fad673f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:17:52.296462 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:17:52.362961 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f39e39-0dc9-4865-aa5e-36b1fad673f9 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:52.367368 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd326fa-2846-4813-b5cd-64b6811f76a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:52.368762 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd326fa-2846-4813-b5cd-64b6811f76a9 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:52.369154 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd326fa-2846-4813-b5cd-64b6811f76a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:52.369536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd326fa-2846-4813-b5cd-64b6811f76a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:52.382325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd326fa-2846-4813-b5cd-64b6811f76a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:52.383996 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd326fa-2846-4813-b5cd-64b6811f76a9 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:52.396779 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1407/2865] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:17:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:17:52.782891 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266609b8-e885-435c-ad7b-1c1a289a03e7 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:17:52.801005 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266609b8-e885-435c-ad7b-1c1a289a03e7 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:17:52.807963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd18a70-a7c5-4288-a356-f34fd9db52e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:17:52.809067 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd18a70-a7c5-4288-a356-f34fd9db52e9 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:17:52.809553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd18a70-a7c5-4288-a356-f34fd9db52e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:17:52.810048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd18a70-a7c5-4288-a356-f34fd9db52e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:17:52.922194 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd18a70-a7c5-4288-a356-f34fd9db52e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:17:52.923893 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd18a70-a7c5-4288-a356-f34fd9db52e9 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:17:52.934528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1459/2866] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:17:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:18:02.464331 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=25 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:18:02.929586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=40 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:18:49.533601 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd326fa-2846-4813-b5cd-64b6811f76a9 service cinder] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:49.648135 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd326fa-2846-4813-b5cd-64b6811f76a9 service cinder] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:49.653899 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:49.654551 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:49.654835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:49.655228 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] PATH_INFO: `/v3/users/b1efe00df1724424816f71332367c46d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:49.657037 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1efe00df1724424816f71332367c46d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:49.660095 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:50.066981 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1efe00df1724424816f71332367c46d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:50.075371 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1efe00df1724424816f71332367c46d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:50.076112 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1efe00df1724424816f71332367c46d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:50.177645 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1efe00df1724424816f71332367c46d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:50.195713 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1efe00df1724424816f71332367c46d'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:50.202773 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1408/2867] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:18:49 2022] DELETE /identity/v3/users/b1efe00df1724424816f71332367c46d => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:18:50.208006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff597e-8a72-4598-90e3-1b3447636596 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:50.208664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff597e-8a72-4598-90e3-1b3447636596 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:50.208664 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff597e-8a72-4598-90e3-1b3447636596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:50.249592 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-35ff597e-8a72-4598-90e3-1b3447636596 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:18:50.250852 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-35ff597e-8a72-4598-90e3-1b3447636596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:18:50.329444 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1460/2868] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:18:50.341980 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:50.343575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:18:50.419955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b1b9a-ff85-478b-ac31-3d04fac82687 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:50.424689 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a9867-5530-48d2-9557-2c1662276735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:50.425221 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a9867-5530-48d2-9557-2c1662276735 service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:50.425540 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a9867-5530-48d2-9557-2c1662276735 service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:50.425817 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a9867-5530-48d2-9557-2c1662276735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:50.439021 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a9867-5530-48d2-9557-2c1662276735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:50.440545 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a9867-5530-48d2-9557-2c1662276735 service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:50.445183 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:18:50.453493 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1409/2869] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:18:51.034000 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff597e-8a72-4598-90e3-1b3447636596 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:51.036472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-35ff597e-8a72-4598-90e3-1b3447636596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:18:51.058371 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff597e-8a72-4598-90e3-1b3447636596 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:51.064834 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:51.065702 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:51.066143 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:51.066577 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] PATH_INFO: `/v3/projects/daa12ae821ff4d50943f51b76140f2e1` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:51.069640 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa12ae821ff4d50943f51b76140f2e1)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:51.072223 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:51.233043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa12ae821ff4d50943f51b76140f2e1', 'request_id': 'req-38283a77-d396-4171-a4ea-0296d4b7bdbc'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:51.240071 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa12ae821ff4d50943f51b76140f2e1', 'request_id': 'req-38283a77-d396-4171-a4ea-0296d4b7bdbc'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:51.354463 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa12ae821ff4d50943f51b76140f2e1', 'request_id': 'req-38283a77-d396-4171-a4ea-0296d4b7bdbc'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:51.370104 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] The token cache is being invalidate because project daa12ae821ff4d50943f51b76140f2e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:18:51.370472 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:51.371274 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:51.372973 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1461/2870] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:18:51 2022] DELETE /identity/v3/projects/daa12ae821ff4d50943f51b76140f2e1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:18:51.377563 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a9867-5530-48d2-9557-2c1662276735 service neutron] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:51.389156 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a9867-5530-48d2-9557-2c1662276735 service neutron] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:51.495585 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:51.496524 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:51.499971 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:51.499971 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] PATH_INFO: `/v3/users/0f9eca0c2aea476ab7d5e533aaf8a5ff` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:51.499971 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9eca0c2aea476ab7d5e533aaf8a5ff)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:51.501014 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:51.746041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9eca0c2aea476ab7d5e533aaf8a5ff'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:51.755831 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9eca0c2aea476ab7d5e533aaf8a5ff'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:51.756786 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9eca0c2aea476ab7d5e533aaf8a5ff'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:51.874541 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9eca0c2aea476ab7d5e533aaf8a5ff'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:51.891829 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-14c3dba2-c2cb-4456-bc3f-a747a3ba8f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9eca0c2aea476ab7d5e533aaf8a5ff'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:51.902470 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1410/2871] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:18:51 2022] DELETE /identity/v3/users/0f9eca0c2aea476ab7d5e533aaf8a5ff => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:18:52.420576 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:52.422279 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:18:52.448389 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38283a77-d396-4171-a4ea-0296d4b7bdbc admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:52.548573 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:52.549553 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:52.550048 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:52.550546 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] PATH_INFO: `/v3/projects/bc287ba0857d45a0982c2f47024eaeef` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:52.552733 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc287ba0857d45a0982c2f47024eaeef)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:52.557448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:52.595829 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:18:52.743080 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc287ba0857d45a0982c2f47024eaeef', 'request_id': 'req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:52.749921 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc287ba0857d45a0982c2f47024eaeef', 'request_id': 'req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:52.835364 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc287ba0857d45a0982c2f47024eaeef', 'request_id': 'req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:52.851846 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] The token cache is being invalidate because project bc287ba0857d45a0982c2f47024eaeef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:18:52.852238 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:52.853068 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ec2e97e6-fd5b-497b-83eb-74259eb14c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:52.854800 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1462/2872] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:18:52 2022] DELETE /identity/v3/projects/bc287ba0857d45a0982c2f47024eaeef => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:18:52.860473 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c5058-77c9-455c-af2d-78f1c72b75ed None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:52.861250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c5058-77c9-455c-af2d-78f1c72b75ed None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:52.861250 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c5058-77c9-455c-af2d-78f1c72b75ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:52.883388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-124c5058-77c9-455c-af2d-78f1c72b75ed None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:18:52.884783 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-124c5058-77c9-455c-af2d-78f1c72b75ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:18:52.961201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1411/2873] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:18:52.969647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a9d3a-6eb1-4494-81fe-a4bcaf29946f None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:52.970080 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a9d3a-6eb1-4494-81fe-a4bcaf29946f None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:52.970473 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a9d3a-6eb1-4494-81fe-a4bcaf29946f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:52.997725 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-cb3a9d3a-6eb1-4494-81fe-a4bcaf29946f None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:18:52.999235 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cb3a9d3a-6eb1-4494-81fe-a4bcaf29946f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:18:53.009496 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1463/2874] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:18:53.014319 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c5058-77c9-455c-af2d-78f1c72b75ed None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:53.025279 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c5058-77c9-455c-af2d-78f1c72b75ed None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:53.030881 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55e7ef-1ffb-4d1f-9df6-b27c7f78687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:53.031794 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55e7ef-1ffb-4d1f-9df6-b27c7f78687c admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:53.032298 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55e7ef-1ffb-4d1f-9df6-b27c7f78687c admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:53.032624 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55e7ef-1ffb-4d1f-9df6-b27c7f78687c admin admin] PATH_INFO: `/v3/domains` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:53.033390 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55e7ef-1ffb-4d1f-9df6-b27c7f78687c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:18:53.033837 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55e7ef-1ffb-4d1f-9df6-b27c7f78687c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:53.035791 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55e7ef-1ffb-4d1f-9df6-b27c7f78687c admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:53.050055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1412/2875] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:18:53.056544 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a9d3a-6eb1-4494-81fe-a4bcaf29946f None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:53.069259 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a9d3a-6eb1-4494-81fe-a4bcaf29946f None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:53.074069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4576c-2f3f-45ab-ad08-1e4e832ad4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:53.074748 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4576c-2f3f-45ab-ad08-1e4e832ad4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:53.075069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4576c-2f3f-45ab-ad08-1e4e832ad4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:53.075479 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4576c-2f3f-45ab-ad08-1e4e832ad4b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:53.076207 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4576c-2f3f-45ab-ad08-1e4e832ad4b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:53.078451 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4576c-2f3f-45ab-ad08-1e4e832ad4b1 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:53.453440 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1464/2876] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:18:53 2022] POST /identity/v3/projects => generated 367 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:18:53.458858 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55e7ef-1ffb-4d1f-9df6-b27c7f78687c admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:53.465835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:18:53.470415 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55e7ef-1ffb-4d1f-9df6-b27c7f78687c admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:53.543819 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe37829-62b8-4cdc-b38b-d943b844b7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:53.544587 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe37829-62b8-4cdc-b38b-d943b844b7f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:53.544914 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe37829-62b8-4cdc-b38b-d943b844b7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:53.545233 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe37829-62b8-4cdc-b38b-d943b844b7f6 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:53.545972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe37829-62b8-4cdc-b38b-d943b844b7f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:53.548303 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe37829-62b8-4cdc-b38b-d943b844b7f6 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:53.716293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1413/2877] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:18:53 2022] POST /identity/v3/users => generated 284 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:18:53.721113 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4576c-2f3f-45ab-ad08-1e4e832ad4b1 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:53.742738 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4576c-2f3f-45ab-ad08-1e4e832ad4b1 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:53.750806 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c83577-f580-42f4-9901-511b101332df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:53.751876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c83577-f580-42f4-9901-511b101332df admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:53.752424 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c83577-f580-42f4-9901-511b101332df admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:53.753132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c83577-f580-42f4-9901-511b101332df admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:53.754368 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c83577-f580-42f4-9901-511b101332df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:53.757378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c83577-f580-42f4-9901-511b101332df admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:53.775569 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1465/2878] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:18:53.780821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe37829-62b8-4cdc-b38b-d943b844b7f6 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:53.797418 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe37829-62b8-4cdc-b38b-d943b844b7f6 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:53.805520 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef7cc8-ecfd-44e9-b116-600c39a5b711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:53.806670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef7cc8-ecfd-44e9-b116-600c39a5b711 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:53.807276 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef7cc8-ecfd-44e9-b116-600c39a5b711 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:53.807778 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef7cc8-ecfd-44e9-b116-600c39a5b711 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:53.809006 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef7cc8-ecfd-44e9-b116-600c39a5b711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:53.812131 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef7cc8-ecfd-44e9-b116-600c39a5b711 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:53.830350 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1414/2879] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:18:53.835644 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c83577-f580-42f4-9901-511b101332df admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:53.846906 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c83577-f580-42f4-9901-511b101332df admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:53.852840 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ea117-5888-4585-b2be-0a9948580035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:53.853695 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ea117-5888-4585-b2be-0a9948580035 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:53.854124 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ea117-5888-4585-b2be-0a9948580035 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:53.854528 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ea117-5888-4585-b2be-0a9948580035 admin admin] PATH_INFO: `/v3/projects/f620004159454ca3aefa0d8c68404b99/users/8eab0b04ddc547ea9c16ac0201f74b14/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:53.878486 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ea117-5888-4585-b2be-0a9948580035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f620004159454ca3aefa0d8c68404b99, user_id=8eab0b04ddc547ea9c16ac0201f74b14, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:53.881383 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ea117-5888-4585-b2be-0a9948580035 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:53.973660 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1466/2880] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:18:53 2022] PUT /identity/v3/projects/f620004159454ca3aefa0d8c68404b99/users/8eab0b04ddc547ea9c16ac0201f74b14/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:18:53.990451 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca88f6-54fc-4380-9664-604133b2d0d5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:53.991049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca88f6-54fc-4380-9664-604133b2d0d5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:53.991049 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca88f6-54fc-4380-9664-604133b2d0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:54.028941 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-9fca88f6-54fc-4380-9664-604133b2d0d5 None None] MFA Rules not processed for user `8eab0b04ddc547ea9c16ac0201f74b14`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:18:54.030219 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-9fca88f6-54fc-4380-9664-604133b2d0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:18:54.113827 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1415/2881] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:18:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:18:54.119669 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ea117-5888-4585-b2be-0a9948580035 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:54.217092 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ea117-5888-4585-b2be-0a9948580035 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:54.225705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad7415-cfbb-4ddd-82e2-c1b2e446b686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:54.226630 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad7415-cfbb-4ddd-82e2-c1b2e446b686 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:54.227105 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad7415-cfbb-4ddd-82e2-c1b2e446b686 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:54.227604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad7415-cfbb-4ddd-82e2-c1b2e446b686 admin admin] PATH_INFO: `/v3/projects` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:54.228780 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad7415-cfbb-4ddd-82e2-c1b2e446b686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:54.232461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad7415-cfbb-4ddd-82e2-c1b2e446b686 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:54.387972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1467/2882] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:18:54 2022] POST /identity/v3/projects => generated 367 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:18:54.393027 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca88f6-54fc-4380-9664-604133b2d0d5 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:54.408361 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca88f6-54fc-4380-9664-604133b2d0d5 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:54.487424 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653ef43-706e-4103-b788-1a2def7839d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:54.487966 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ef43-706e-4103-b788-1a2def7839d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:54.488369 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ef43-706e-4103-b788-1a2def7839d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:54.488670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ef43-706e-4103-b788-1a2def7839d2 admin admin] PATH_INFO: `/v3/users` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:54.489363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653ef43-706e-4103-b788-1a2def7839d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:54.492056 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653ef43-706e-4103-b788-1a2def7839d2 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:54.664055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1416/2883] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:18:54 2022] POST /identity/v3/users => generated 290 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:18:54.668566 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad7415-cfbb-4ddd-82e2-c1b2e446b686 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:54.681201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad7415-cfbb-4ddd-82e2-c1b2e446b686 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:54.686941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499bf5e-9584-4b8b-9ccd-9c85e8d1c30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:54.687938 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499bf5e-9584-4b8b-9ccd-9c85e8d1c30b admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:54.688382 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499bf5e-9584-4b8b-9ccd-9c85e8d1c30b admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:54.688708 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499bf5e-9584-4b8b-9ccd-9c85e8d1c30b admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:54.689420 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499bf5e-9584-4b8b-9ccd-9c85e8d1c30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:54.691232 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499bf5e-9584-4b8b-9ccd-9c85e8d1c30b admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:54.705116 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1468/2884] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:18:54.719979 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653ef43-706e-4103-b788-1a2def7839d2 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:54.727240 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:18:54.733577 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653ef43-706e-4103-b788-1a2def7839d2 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:54.740142 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3329c-61ed-49c4-957c-ca24a4351bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:54.741479 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3329c-61ed-49c4-957c-ca24a4351bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:54.741767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3329c-61ed-49c4-957c-ca24a4351bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:54.742038 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3329c-61ed-49c4-957c-ca24a4351bbd admin admin] PATH_INFO: `/v3/projects/2acd131e2e85481e9824878d95b9b570/users/cfd93f492d344b8b805438a17a8e67f6/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:54.762181 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3329c-61ed-49c4-957c-ca24a4351bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2acd131e2e85481e9824878d95b9b570, user_id=cfd93f492d344b8b805438a17a8e67f6, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:54.766182 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3329c-61ed-49c4-957c-ca24a4351bbd admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:54.922091 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1417/2885] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:18:54 2022] PUT /identity/v3/projects/2acd131e2e85481e9824878d95b9b570/users/cfd93f492d344b8b805438a17a8e67f6/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:18:54.928367 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499bf5e-9584-4b8b-9ccd-9c85e8d1c30b admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:54.938818 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499bf5e-9584-4b8b-9ccd-9c85e8d1c30b admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:54.954285 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:18:55.005675 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a760501-3f3a-42eb-b16b-1e9c70235fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:55.006691 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a760501-3f3a-42eb-b16b-1e9c70235fca admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:55.006980 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a760501-3f3a-42eb-b16b-1e9c70235fca admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:55.007316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a760501-3f3a-42eb-b16b-1e9c70235fca admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:55.007963 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a760501-3f3a-42eb-b16b-1e9c70235fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:55.009702 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a760501-3f3a-42eb-b16b-1e9c70235fca admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:55.023271 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1469/2886] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:18:55.028472 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3329c-61ed-49c4-957c-ca24a4351bbd admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:55.040626 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3329c-61ed-49c4-957c-ca24a4351bbd admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:55.045724 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e34bb2-d86c-4107-b2bd-8a895a8c3a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:55.046566 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e34bb2-d86c-4107-b2bd-8a895a8c3a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:55.046777 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e34bb2-d86c-4107-b2bd-8a895a8c3a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:55.047055 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e34bb2-d86c-4107-b2bd-8a895a8c3a84 admin admin] PATH_INFO: `/v3/projects/2acd131e2e85481e9824878d95b9b570/users/cfd93f492d344b8b805438a17a8e67f6/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:55.049227 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e34bb2-d86c-4107-b2bd-8a895a8c3a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2acd131e2e85481e9824878d95b9b570, user_id=cfd93f492d344b8b805438a17a8e67f6, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:55.051955 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e34bb2-d86c-4107-b2bd-8a895a8c3a84 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:55.075958 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1418/2887] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:18:55 2022] PUT /identity/v3/projects/2acd131e2e85481e9824878d95b9b570/users/cfd93f492d344b8b805438a17a8e67f6/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:18:55.088453 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf1ef2-4675-4868-af8d-ef4ee0eca698 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:55.088847 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf1ef2-4675-4868-af8d-ef4ee0eca698 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:55.089174 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf1ef2-4675-4868-af8d-ef4ee0eca698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:55.124557 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-45cf1ef2-4675-4868-af8d-ef4ee0eca698 None None] MFA Rules not processed for user `cfd93f492d344b8b805438a17a8e67f6`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:18:55.128294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-45cf1ef2-4675-4868-af8d-ef4ee0eca698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:18:55.213608 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1470/2888] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:18:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:18:55.226839 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e34bb2-d86c-4107-b2bd-8a895a8c3a84 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:55.228774 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-06e34bb2-d86c-4107-b2bd-8a895a8c3a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:18:55.301711 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e34bb2-d86c-4107-b2bd-8a895a8c3a84 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:55.306338 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535572b-413a-40e0-8550-e07a8e08be7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:55.306977 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535572b-413a-40e0-8550-e07a8e08be7b service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:55.307266 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535572b-413a-40e0-8550-e07a8e08be7b service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:55.307544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535572b-413a-40e0-8550-e07a8e08be7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:55.318307 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535572b-413a-40e0-8550-e07a8e08be7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:55.319835 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535572b-413a-40e0-8550-e07a8e08be7b service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:55.334893 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1419/2889] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:18:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:18:59.354258 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf1ef2-4675-4868-af8d-ef4ee0eca698 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:18:59.469281 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf1ef2-4675-4868-af8d-ef4ee0eca698 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:18:59.477417 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:18:59.478359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:59.478788 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:59.479260 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] PATH_INFO: `/v3/users/8eab0b04ddc547ea9c16ac0201f74b14` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:59.481006 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eab0b04ddc547ea9c16ac0201f74b14)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:18:59.483632 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:18:59.729061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:18:59.736946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eab0b04ddc547ea9c16ac0201f74b14'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:59.737744 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eab0b04ddc547ea9c16ac0201f74b14'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:59.747822 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eab0b04ddc547ea9c16ac0201f74b14'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:59.748668 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eab0b04ddc547ea9c16ac0201f74b14'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:59.839882 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eab0b04ddc547ea9c16ac0201f74b14'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:18:59.860872 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1471/2890] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:18:59 2022] DELETE /identity/v3/users/8eab0b04ddc547ea9c16ac0201f74b14 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:18:59.868816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67afed-a06a-48df-bc45-86be55f296e4 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:18:59.869557 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67afed-a06a-48df-bc45-86be55f296e4 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:18:59.869789 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67afed-a06a-48df-bc45-86be55f296e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:18:59.903766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-5d67afed-a06a-48df-bc45-86be55f296e4 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:18:59.906041 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5d67afed-a06a-48df-bc45-86be55f296e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:00.000912 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1420/2891] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:19:00.012610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:00.014346 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:00.084941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c9e58-9bc3-4051-899a-b406b786c79d admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:00.089164 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a0840-e005-4902-9de4-c774bc5363c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:00.089839 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a0840-e005-4902-9de4-c774bc5363c1 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:00.090125 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a0840-e005-4902-9de4-c774bc5363c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:00.090425 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a0840-e005-4902-9de4-c774bc5363c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:00.101199 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a0840-e005-4902-9de4-c774bc5363c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:00.102765 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a0840-e005-4902-9de4-c774bc5363c1 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:00.116044 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1472/2892] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:19:00.686163 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67afed-a06a-48df-bc45-86be55f296e4 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:00.688826 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-5d67afed-a06a-48df-bc45-86be55f296e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:00.711544 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67afed-a06a-48df-bc45-86be55f296e4 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:00.719117 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:00.725797 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:00.726206 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:00.726529 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] PATH_INFO: `/v3/projects/f620004159454ca3aefa0d8c68404b99` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:00.728864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f620004159454ca3aefa0d8c68404b99)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:00.731497 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:00.877217 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f620004159454ca3aefa0d8c68404b99', 'request_id': 'req-12304800-194a-4ec4-bfb0-24bb6ea2989b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:00.884666 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f620004159454ca3aefa0d8c68404b99', 'request_id': 'req-12304800-194a-4ec4-bfb0-24bb6ea2989b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:00.892069 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f620004159454ca3aefa0d8c68404b99', 'request_id': 'req-12304800-194a-4ec4-bfb0-24bb6ea2989b'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:00.894731 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:01.018102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] The token cache is being invalidate because project f620004159454ca3aefa0d8c68404b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:19:01.018464 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:01.019293 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:01.021274 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1421/2893] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:19:00 2022] DELETE /identity/v3/projects/f620004159454ca3aefa0d8c68404b99 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:19:01.026615 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a0840-e005-4902-9de4-c774bc5363c1 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:01.037999 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a0840-e005-4902-9de4-c774bc5363c1 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:01.105518 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:01.106167 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:01.106448 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:01.106726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] PATH_INFO: `/v3/users/cfd93f492d344b8b805438a17a8e67f6` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:01.107989 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd93f492d344b8b805438a17a8e67f6)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:01.109988 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:01.378231 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd93f492d344b8b805438a17a8e67f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:01.378928 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd93f492d344b8b805438a17a8e67f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:01.388203 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd93f492d344b8b805438a17a8e67f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:01.388969 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd93f492d344b8b805438a17a8e67f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:01.393665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:01.503348 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-0b22c209-02dd-4730-b205-d8f84ae0c6fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd93f492d344b8b805438a17a8e67f6'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:01.524525 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1473/2894] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:19:01 2022] DELETE /identity/v3/users/cfd93f492d344b8b805438a17a8e67f6 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:19:02.088421 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:02.090397 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:02.111554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12304800-194a-4ec4-bfb0-24bb6ea2989b admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:02.189523 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:02.190283 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:02.190683 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:02.191158 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] PATH_INFO: `/v3/projects/2acd131e2e85481e9824878d95b9b570` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:02.192648 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2acd131e2e85481e9824878d95b9b570)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:02.196024 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:02.355960 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2acd131e2e85481e9824878d95b9b570', 'request_id': 'req-76352aed-cd94-4a45-baad-37beb9f199e0'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:02.363194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2acd131e2e85481e9824878d95b9b570', 'request_id': 'req-76352aed-cd94-4a45-baad-37beb9f199e0'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:02.370224 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2acd131e2e85481e9824878d95b9b570', 'request_id': 'req-76352aed-cd94-4a45-baad-37beb9f199e0'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:02.474207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] The token cache is being invalidate because project 2acd131e2e85481e9824878d95b9b570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:19:02.474502 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:02.475126 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-76352aed-cd94-4a45-baad-37beb9f199e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:02.476757 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1422/2895] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:19:02 2022] DELETE /identity/v3/projects/2acd131e2e85481e9824878d95b9b570 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:19:02.481977 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955cd17-81a6-4cf7-beb4-465e8d8441a6 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:02.482565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955cd17-81a6-4cf7-beb4-465e8d8441a6 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:02.482565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955cd17-81a6-4cf7-beb4-465e8d8441a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:02.504945 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-f955cd17-81a6-4cf7-beb4-465e8d8441a6 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:19:02.507230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-f955cd17-81a6-4cf7-beb4-465e8d8441a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:02.587932 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1474/2896] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:19:02.598356 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c14b08-0984-490b-b1d4-1f1766f50d5e None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:02.598843 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c14b08-0984-490b-b1d4-1f1766f50d5e None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:02.599327 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c14b08-0984-490b-b1d4-1f1766f50d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:02.627109 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-f1c14b08-0984-490b-b1d4-1f1766f50d5e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:19:02.628658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-f1c14b08-0984-490b-b1d4-1f1766f50d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:02.637517 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1423/2897] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:19:02.642110 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955cd17-81a6-4cf7-beb4-465e8d8441a6 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:02.654375 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955cd17-81a6-4cf7-beb4-465e8d8441a6 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:02.661787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119677d-ce2c-4910-bf21-57f5c7f21814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:02.662830 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119677d-ce2c-4910-bf21-57f5c7f21814 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:02.663336 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119677d-ce2c-4910-bf21-57f5c7f21814 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:02.663821 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119677d-ce2c-4910-bf21-57f5c7f21814 admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:02.665082 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119677d-ce2c-4910-bf21-57f5c7f21814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:19:02.665624 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119677d-ce2c-4910-bf21-57f5c7f21814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:02.668635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119677d-ce2c-4910-bf21-57f5c7f21814 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:02.690783 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1475/2898] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:19:02.698101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c14b08-0984-490b-b1d4-1f1766f50d5e None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:02.708236 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c14b08-0984-490b-b1d4-1f1766f50d5e None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:02.713385 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ff8df-9dc4-4331-b138-8b185f4b6e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:02.714004 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ff8df-9dc4-4331-b138-8b185f4b6e98 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:02.714282 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ff8df-9dc4-4331-b138-8b185f4b6e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:02.714556 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ff8df-9dc4-4331-b138-8b185f4b6e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:02.715207 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ff8df-9dc4-4331-b138-8b185f4b6e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:02.717194 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ff8df-9dc4-4331-b138-8b185f4b6e98 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:02.831105 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1424/2899] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:19:02 2022] POST /identity/v3/projects => generated 409 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:19:02.838276 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119677d-ce2c-4910-bf21-57f5c7f21814 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:02.856078 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119677d-ce2c-4910-bf21-57f5c7f21814 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:02.943863 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6ac02-470b-4f0d-8aa2-d9c72fb2ccb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:02.944512 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6ac02-470b-4f0d-8aa2-d9c72fb2ccb9 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:02.944793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6ac02-470b-4f0d-8aa2-d9c72fb2ccb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:02.945061 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6ac02-470b-4f0d-8aa2-d9c72fb2ccb9 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:02.945754 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6ac02-470b-4f0d-8aa2-d9c72fb2ccb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:02.947886 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6ac02-470b-4f0d-8aa2-d9c72fb2ccb9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:03.082410 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1476/2900] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:19:02 2022] POST /identity/v3/users => generated 305 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:19:03.087273 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ff8df-9dc4-4331-b138-8b185f4b6e98 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:03.102201 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ff8df-9dc4-4331-b138-8b185f4b6e98 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:03.109936 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47b443-c8f7-421d-aad9-4ae7f96bfc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:03.110612 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47b443-c8f7-421d-aad9-4ae7f96bfc82 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:03.110934 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47b443-c8f7-421d-aad9-4ae7f96bfc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:03.111259 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47b443-c8f7-421d-aad9-4ae7f96bfc82 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:03.111974 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47b443-c8f7-421d-aad9-4ae7f96bfc82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:03.113864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47b443-c8f7-421d-aad9-4ae7f96bfc82 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:03.126023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1425/2901] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:19:03.131091 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6ac02-470b-4f0d-8aa2-d9c72fb2ccb9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:03.142153 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6ac02-470b-4f0d-8aa2-d9c72fb2ccb9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:03.149244 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93278d-dc9a-462b-b304-d515c0659560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:03.150284 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93278d-dc9a-462b-b304-d515c0659560 admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:03.150778 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93278d-dc9a-462b-b304-d515c0659560 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:03.151295 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93278d-dc9a-462b-b304-d515c0659560 admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:03.152519 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93278d-dc9a-462b-b304-d515c0659560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:03.155192 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93278d-dc9a-462b-b304-d515c0659560 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:03.176513 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1477/2902] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:19:03.183011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47b443-c8f7-421d-aad9-4ae7f96bfc82 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:03.200988 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47b443-c8f7-421d-aad9-4ae7f96bfc82 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:03.208767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba720be-7c1f-47b0-a6c0-8b738df26f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:03.209889 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba720be-7c1f-47b0-a6c0-8b738df26f64 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:03.210383 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba720be-7c1f-47b0-a6c0-8b738df26f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:03.210876 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba720be-7c1f-47b0-a6c0-8b738df26f64 admin admin] PATH_INFO: `/v3/projects/6227f36517ff40cc85be03a927e94a13/users/8a1c20a91908461cbc746e878c1449f4/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:03.232065 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba720be-7c1f-47b0-a6c0-8b738df26f64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6227f36517ff40cc85be03a927e94a13, user_id=8a1c20a91908461cbc746e878c1449f4, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:03.234897 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba720be-7c1f-47b0-a6c0-8b738df26f64 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:03.355433 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1426/2903] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:19:03 2022] PUT /identity/v3/projects/6227f36517ff40cc85be03a927e94a13/users/8a1c20a91908461cbc746e878c1449f4/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:19:03.370837 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0b48a-c4db-47c9-92e4-479d74034e6a None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:03.371208 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0b48a-c4db-47c9-92e4-479d74034e6a None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:03.371378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0b48a-c4db-47c9-92e4-479d74034e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:03.403993 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-60f0b48a-c4db-47c9-92e4-479d74034e6a None None] MFA Rules not processed for user `8a1c20a91908461cbc746e878c1449f4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:19:03.405524 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-60f0b48a-c4db-47c9-92e4-479d74034e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:03.446287 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:03.485277 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1478/2904] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:19:03 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:19:03.490359 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba720be-7c1f-47b0-a6c0-8b738df26f64 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:03.514199 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:03.563598 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba720be-7c1f-47b0-a6c0-8b738df26f64 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:03.569660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ac620-a57e-4d9e-b2ee-b3a8a4cb3a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:03.570345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ac620-a57e-4d9e-b2ee-b3a8a4cb3a1a admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:03.570670 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ac620-a57e-4d9e-b2ee-b3a8a4cb3a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:03.571123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ac620-a57e-4d9e-b2ee-b3a8a4cb3a1a admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:03.571937 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ac620-a57e-4d9e-b2ee-b3a8a4cb3a1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:03.574102 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ac620-a57e-4d9e-b2ee-b3a8a4cb3a1a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:03.700096 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1427/2905] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:19:03 2022] POST /identity/v3/projects => generated 409 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:19:03.705036 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0b48a-c4db-47c9-92e4-479d74034e6a None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:03.718380 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0b48a-c4db-47c9-92e4-479d74034e6a None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:03.792347 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce341534-ed4e-4bfe-ac2e-f5b282e400f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:03.793035 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce341534-ed4e-4bfe-ac2e-f5b282e400f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:03.793501 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce341534-ed4e-4bfe-ac2e-f5b282e400f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:03.793934 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce341534-ed4e-4bfe-ac2e-f5b282e400f9 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:03.794741 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce341534-ed4e-4bfe-ac2e-f5b282e400f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:03.797001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce341534-ed4e-4bfe-ac2e-f5b282e400f9 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:03.906228 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1479/2906] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:19:03 2022] POST /identity/v3/users => generated 311 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:19:03.919225 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ac620-a57e-4d9e-b2ee-b3a8a4cb3a1a admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:03.923816 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ac620-a57e-4d9e-b2ee-b3a8a4cb3a1a admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:03.930071 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fab9b5-fe42-4edc-baa1-ed5a8d7dfca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:03.930815 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fab9b5-fe42-4edc-baa1-ed5a8d7dfca8 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:03.931211 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fab9b5-fe42-4edc-baa1-ed5a8d7dfca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:03.931536 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fab9b5-fe42-4edc-baa1-ed5a8d7dfca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:03.932394 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fab9b5-fe42-4edc-baa1-ed5a8d7dfca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:03.934127 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fab9b5-fe42-4edc-baa1-ed5a8d7dfca8 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:03.946652 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1428/2907] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:19:03.951982 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce341534-ed4e-4bfe-ac2e-f5b282e400f9 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:03.964241 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce341534-ed4e-4bfe-ac2e-f5b282e400f9 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:03.969827 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253cfd1-450a-410e-bbcd-dc8874d20aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:03.970705 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253cfd1-450a-410e-bbcd-dc8874d20aae admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:03.971096 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253cfd1-450a-410e-bbcd-dc8874d20aae admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:03.971554 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253cfd1-450a-410e-bbcd-dc8874d20aae admin admin] PATH_INFO: `/v3/projects/2eb73036aade4647b05413372328e0b2/users/6b574ef56feb49ce9ff40fbcfbf80201/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:03.991175 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253cfd1-450a-410e-bbcd-dc8874d20aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb73036aade4647b05413372328e0b2, user_id=6b574ef56feb49ce9ff40fbcfbf80201, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:03.994791 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253cfd1-450a-410e-bbcd-dc8874d20aae admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:04.110288 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1480/2908] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:19:03 2022] PUT /identity/v3/projects/2eb73036aade4647b05413372328e0b2/users/6b574ef56feb49ce9ff40fbcfbf80201/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:19:04.115823 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fab9b5-fe42-4edc-baa1-ed5a8d7dfca8 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:04.126905 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fab9b5-fe42-4edc-baa1-ed5a8d7dfca8 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:04.195885 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8744cf6-b704-4663-a331-8fbed403e5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:04.196571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8744cf6-b704-4663-a331-8fbed403e5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:04.196851 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8744cf6-b704-4663-a331-8fbed403e5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:04.197123 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8744cf6-b704-4663-a331-8fbed403e5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:04.197821 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8744cf6-b704-4663-a331-8fbed403e5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:04.199498 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8744cf6-b704-4663-a331-8fbed403e5f1 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:04.211736 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1429/2909] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:19:04.216392 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253cfd1-450a-410e-bbcd-dc8874d20aae admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:04.227894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253cfd1-450a-410e-bbcd-dc8874d20aae admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:04.232955 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75b0f3-b647-4e85-931c-222c15749613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:04.233639 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75b0f3-b647-4e85-931c-222c15749613 admin admin] REQUEST_METHOD: `PUT` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:04.233919 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75b0f3-b647-4e85-931c-222c15749613 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:04.234196 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75b0f3-b647-4e85-931c-222c15749613 admin admin] PATH_INFO: `/v3/projects/2eb73036aade4647b05413372328e0b2/users/6b574ef56feb49ce9ff40fbcfbf80201/roles/fce271509b2c4922bd9cf3f987c9e34d` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:04.236289 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75b0f3-b647-4e85-931c-222c15749613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb73036aade4647b05413372328e0b2, user_id=6b574ef56feb49ce9ff40fbcfbf80201, role_id=fce271509b2c4922bd9cf3f987c9e34d)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:04.239009 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75b0f3-b647-4e85-931c-222c15749613 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:04.251707 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1481/2910] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:19:04 2022] PUT /identity/v3/projects/2eb73036aade4647b05413372328e0b2/users/6b574ef56feb49ce9ff40fbcfbf80201/roles/fce271509b2c4922bd9cf3f987c9e34d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:19:04.264050 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b28b99-91bf-4d17-bd4c-430cd8d215e9 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:04.264375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b28b99-91bf-4d17-bd4c-430cd8d215e9 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:04.264660 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b28b99-91bf-4d17-bd4c-430cd8d215e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:04.296849 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-73b28b99-91bf-4d17-bd4c-430cd8d215e9 None None] MFA Rules not processed for user `6b574ef56feb49ce9ff40fbcfbf80201`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:19:04.298202 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-73b28b99-91bf-4d17-bd4c-430cd8d215e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:04.381664 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1430/2911] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:19:04 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:19:04.394205 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75b0f3-b647-4e85-931c-222c15749613 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:04.396115 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-cd75b0f3-b647-4e85-931c-222c15749613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:04.466552 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75b0f3-b647-4e85-931c-222c15749613 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:04.470604 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe885dd-eba1-4710-aa94-b4342e578373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:04.471233 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe885dd-eba1-4710-aa94-b4342e578373 service cinder] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:04.471511 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe885dd-eba1-4710-aa94-b4342e578373 service cinder] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:04.471787 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe885dd-eba1-4710-aa94-b4342e578373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:04.483252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe885dd-eba1-4710-aa94-b4342e578373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:04.484764 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe885dd-eba1-4710-aa94-b4342e578373 service cinder] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:04.495647 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1482/2912] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:19:04 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:19:04.960805 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b28b99-91bf-4d17-bd4c-430cd8d215e9 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:04.972929 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b28b99-91bf-4d17-bd4c-430cd8d215e9 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:04.976950 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da5326-5ac4-4a47-8090-eafb1bf79324 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '496a2e8bf591437888ea82161ca4d5ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:04.977569 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da5326-5ac4-4a47-8090-eafb1bf79324 service cinder] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:04.977850 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da5326-5ac4-4a47-8090-eafb1bf79324 service cinder] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:04.978122 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da5326-5ac4-4a47-8090-eafb1bf79324 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:04.990695 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:05.056714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da5326-5ac4-4a47-8090-eafb1bf79324 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:05.056714 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da5326-5ac4-4a47-8090-eafb1bf79324 service cinder] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:05.066093 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1431/2913] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:19:04 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:19:14.490894 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:15.074818 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=22 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:16.119394 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe885dd-eba1-4710-aa94-b4342e578373 service cinder] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:16.223075 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe885dd-eba1-4710-aa94-b4342e578373 service cinder] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:16.235763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:16.235763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:16.235763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:16.235763 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] PATH_INFO: `/v3/users/8a1c20a91908461cbc746e878c1449f4` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:16.236442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1c20a91908461cbc746e878c1449f4)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:16.236442 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:16.452063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1c20a91908461cbc746e878c1449f4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:16.452807 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1c20a91908461cbc746e878c1449f4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:16.462518 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1c20a91908461cbc746e878c1449f4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:16.463342 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1c20a91908461cbc746e878c1449f4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:16.542888 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1c20a91908461cbc746e878c1449f4'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:16.566230 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1483/2914] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:19:16 2022] DELETE /identity/v3/users/8a1c20a91908461cbc746e878c1449f4 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:19:16.571391 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402784fc-9588-4f9a-81f1-8d4d22491aaa None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:16.572011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402784fc-9588-4f9a-81f1-8d4d22491aaa None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:16.572011 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402784fc-9588-4f9a-81f1-8d4d22491aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:16.609710 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-402784fc-9588-4f9a-81f1-8d4d22491aaa None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:19:16.611222 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-402784fc-9588-4f9a-81f1-8d4d22491aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:16.683375 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1432/2915] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:19:16.695043 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:16.696956 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:16.818802 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:16.827616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e7565-7df2-4163-9a22-088aa6931cef admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:16.832941 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d64246-cf7e-4cf2-bb0d-1bf843dd4dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:16.834176 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d64246-cf7e-4cf2-bb0d-1bf843dd4dee service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:16.834461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d64246-cf7e-4cf2-bb0d-1bf843dd4dee service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:16.834742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d64246-cf7e-4cf2-bb0d-1bf843dd4dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:16.845586 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d64246-cf7e-4cf2-bb0d-1bf843dd4dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:16.847111 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d64246-cf7e-4cf2-bb0d-1bf843dd4dee service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:16.860797 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1484/2916] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:19:17.430864 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402784fc-9588-4f9a-81f1-8d4d22491aaa None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:17.432522 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-402784fc-9588-4f9a-81f1-8d4d22491aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:17.448575 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402784fc-9588-4f9a-81f1-8d4d22491aaa None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:17.455972 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:17.457028 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:17.457554 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:17.458058 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] PATH_INFO: `/v3/projects/6227f36517ff40cc85be03a927e94a13` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:17.460600 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6227f36517ff40cc85be03a927e94a13)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:17.463388 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:17.598819 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6227f36517ff40cc85be03a927e94a13', 'request_id': 'req-edba724a-0bb0-4ef9-9906-fe340e710f5e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:17.605614 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6227f36517ff40cc85be03a927e94a13', 'request_id': 'req-edba724a-0bb0-4ef9-9906-fe340e710f5e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:17.613661 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6227f36517ff40cc85be03a927e94a13', 'request_id': 'req-edba724a-0bb0-4ef9-9906-fe340e710f5e'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:17.681271 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] The token cache is being invalidate because project 6227f36517ff40cc85be03a927e94a13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:19:17.681532 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:17.682155 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:17.683740 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1433/2917] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:19:17 2022] DELETE /identity/v3/projects/6227f36517ff40cc85be03a927e94a13 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:19:17.689610 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d64246-cf7e-4cf2-bb0d-1bf843dd4dee service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:17.702326 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d64246-cf7e-4cf2-bb0d-1bf843dd4dee service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:17.785058 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:17.786045 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:17.786489 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:17.786939 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] PATH_INFO: `/v3/users/6b574ef56feb49ce9ff40fbcfbf80201` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:17.789014 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b574ef56feb49ce9ff40fbcfbf80201)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:17.792574 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:18.210984 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b574ef56feb49ce9ff40fbcfbf80201'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:18.212842 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b574ef56feb49ce9ff40fbcfbf80201'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:18.223704 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b574ef56feb49ce9ff40fbcfbf80201'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:18.226132 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b574ef56feb49ce9ff40fbcfbf80201'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:18.342808 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-f2a884a1-d8df-4756-9188-25ad2a4af7c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b574ef56feb49ce9ff40fbcfbf80201'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:18.368781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1485/2918] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:19:17 2022] DELETE /identity/v3/users/6b574ef56feb49ce9ff40fbcfbf80201 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:19:19.039882 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:19.049825 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:19.064571 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba724a-0bb0-4ef9-9906-fe340e710f5e admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:19.140855 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:19.141619 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:19.142021 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:19.142408 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] PATH_INFO: `/v3/projects/2eb73036aade4647b05413372328e0b2` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:19.144003 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb73036aade4647b05413372328e0b2)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:19.146387 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:19.165601 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:19.338200 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb73036aade4647b05413372328e0b2', 'request_id': 'req-d427419f-54c5-4f80-9b76-344b3ed1d85a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:19.346178 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb73036aade4647b05413372328e0b2', 'request_id': 'req-d427419f-54c5-4f80-9b76-344b3ed1d85a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:19.354597 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb73036aade4647b05413372328e0b2', 'request_id': 'req-d427419f-54c5-4f80-9b76-344b3ed1d85a'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:19.485363 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] The token cache is being invalidate because project 2eb73036aade4647b05413372328e0b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:19:19.485672 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:19.486409 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-d427419f-54c5-4f80-9b76-344b3ed1d85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:19.488078 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1434/2919] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:19:19 2022] DELETE /identity/v3/projects/2eb73036aade4647b05413372328e0b2 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:19:19.493439 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523225-8c5a-4f43-97ea-2eccaf31445e None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:19.495505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523225-8c5a-4f43-97ea-2eccaf31445e None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:19.495505 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523225-8c5a-4f43-97ea-2eccaf31445e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:19.517542 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-9e523225-8c5a-4f43-97ea-2eccaf31445e None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:19:19.518821 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-9e523225-8c5a-4f43-97ea-2eccaf31445e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:19.589909 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1486/2920] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:19:19.600766 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403156-d20e-46d1-b0fb-b12aa334dcda None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:19.601238 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403156-d20e-46d1-b0fb-b12aa334dcda None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:19.601617 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403156-d20e-46d1-b0fb-b12aa334dcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:19.625203 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-3a403156-d20e-46d1-b0fb-b12aa334dcda None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:19:19.626564 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-3a403156-d20e-46d1-b0fb-b12aa334dcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:19.635909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1435/2921] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:19:19.641693 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e523225-8c5a-4f43-97ea-2eccaf31445e None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:19.658090 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e523225-8c5a-4f43-97ea-2eccaf31445e None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:19.665742 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abf3a2-4312-4e94-aa96-6d4805732f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:19.666793 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abf3a2-4312-4e94-aa96-6d4805732f2e admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:19.667251 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abf3a2-4312-4e94-aa96-6d4805732f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:19.667692 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abf3a2-4312-4e94-aa96-6d4805732f2e admin admin] PATH_INFO: `/v3/domains` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:19.668918 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abf3a2-4312-4e94-aa96-6d4805732f2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:19:19.669461 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abf3a2-4312-4e94-aa96-6d4805732f2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:19.672590 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abf3a2-4312-4e94-aa96-6d4805732f2e admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:19.677063 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:19.689159 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1487/2922] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Oct 4 12:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:19:19.696469 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403156-d20e-46d1-b0fb-b12aa334dcda None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:19.706738 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403156-d20e-46d1-b0fb-b12aa334dcda None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:19.713973 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e9ac5-c16b-4921-8783-2266266222f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:19.714739 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e9ac5-c16b-4921-8783-2266266222f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:19.715138 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e9ac5-c16b-4921-8783-2266266222f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:19.715519 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e9ac5-c16b-4921-8783-2266266222f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:19.716345 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e9ac5-c16b-4921-8783-2266266222f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:19.718547 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e9ac5-c16b-4921-8783-2266266222f2 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:19.819991 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1436/2923] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Oct 4 12:19:19 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:19:19.827187 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abf3a2-4312-4e94-aa96-6d4805732f2e admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:19.846294 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abf3a2-4312-4e94-aa96-6d4805732f2e admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:19.920162 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63ce40-fa47-40f5-84b9-9fde3f57d4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:19.920938 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63ce40-fa47-40f5-84b9-9fde3f57d4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:19.921328 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63ce40-fa47-40f5-84b9-9fde3f57d4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:19.921717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63ce40-fa47-40f5-84b9-9fde3f57d4a8 admin admin] PATH_INFO: `/v3/users` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:19.922558 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63ce40-fa47-40f5-84b9-9fde3f57d4a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:19.925075 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63ce40-fa47-40f5-84b9-9fde3f57d4a8 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:20.027312 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1488/2924] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Oct 4 12:19:19 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:19:20.032392 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e9ac5-c16b-4921-8783-2266266222f2 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:20.047538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e9ac5-c16b-4921-8783-2266266222f2 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:20.054767 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d022318-a56d-4a33-b305-0a67cacf376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:20.058231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d022318-a56d-4a33-b305-0a67cacf376e admin admin] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:20.058231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d022318-a56d-4a33-b305-0a67cacf376e admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:20.058231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d022318-a56d-4a33-b305-0a67cacf376e admin admin] PATH_INFO: `/v3/roles` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:20.058231 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d022318-a56d-4a33-b305-0a67cacf376e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:20.060100 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d022318-a56d-4a33-b305-0a67cacf376e admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:20.080507 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1437/2925] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:19:20.085635 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63ce40-fa47-40f5-84b9-9fde3f57d4a8 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:20.097464 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63ce40-fa47-40f5-84b9-9fde3f57d4a8 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:20.103094 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f092cb-4350-498c-b6b0-580e1fd3f30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:20.103724 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f092cb-4350-498c-b6b0-580e1fd3f30f admin admin] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:20.104001 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f092cb-4350-498c-b6b0-580e1fd3f30f admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:20.104316 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f092cb-4350-498c-b6b0-580e1fd3f30f admin admin] PATH_INFO: `/v3/roles` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:20.104972 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f092cb-4350-498c-b6b0-580e1fd3f30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:20.106619 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f092cb-4350-498c-b6b0-580e1fd3f30f admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:20.118907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1489/2926] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Oct 4 12:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:19:20.126340 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d022318-a56d-4a33-b305-0a67cacf376e admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:20.144119 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d022318-a56d-4a33-b305-0a67cacf376e admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:20.152658 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6b114-8fc3-4df2-8c59-e13df61ab440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:20.153758 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6b114-8fc3-4df2-8c59-e13df61ab440 admin admin] REQUEST_METHOD: `PUT` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:20.154213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6b114-8fc3-4df2-8c59-e13df61ab440 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:20.154687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6b114-8fc3-4df2-8c59-e13df61ab440 admin admin] PATH_INFO: `/v3/projects/af1931b378b64fdebd222ed54491369c/users/b46713e9dbf44c87b84c8ac819bea55e/roles/3f0c34a1f93e49adb1757f315333cc48` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:20.183282 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6b114-8fc3-4df2-8c59-e13df61ab440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1931b378b64fdebd222ed54491369c, user_id=b46713e9dbf44c87b84c8ac819bea55e, role_id=3f0c34a1f93e49adb1757f315333cc48)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:20.189101 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6b114-8fc3-4df2-8c59-e13df61ab440 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:20.276023 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1438/2927] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Oct 4 12:19:20 2022] PUT /identity/v3/projects/af1931b378b64fdebd222ed54491369c/users/b46713e9dbf44c87b84c8ac819bea55e/roles/3f0c34a1f93e49adb1757f315333cc48 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:19:20.287565 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35302812-2f89-413f-801d-3461657aa456 None None] REQUEST_METHOD: `POST` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:20.287936 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35302812-2f89-413f-801d-3461657aa456 None None] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:20.288267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35302812-2f89-413f-801d-3461657aa456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:20.321583 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.auth.core [None req-35302812-2f89-413f-801d-3461657aa456 None None] MFA Rules not processed for user `b46713e9dbf44c87b84c8ac819bea55e`. Rule list: `[]` (Enabled: `True`). {{(pid=93212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:19:20.322878 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-35302812-2f89-413f-801d-3461657aa456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:20.400267 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1490/2928] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:19:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:19:20.416953 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6b114-8fc3-4df2-8c59-e13df61ab440 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:20.423191 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-50a6b114-8fc3-4df2-8c59-e13df61ab440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:20.499685 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6b114-8fc3-4df2-8c59-e13df61ab440 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:20.504646 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe31c84-d1af-4bf5-9b21-c888cdc976e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454fb811f6e34e1d881d4948ede058b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:20.505968 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe31c84-d1af-4bf5-9b21-c888cdc976e9 service glance] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:20.506329 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe31c84-d1af-4bf5-9b21-c888cdc976e9 service glance] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:20.506677 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe31c84-d1af-4bf5-9b21-c888cdc976e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:20.517801 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe31c84-d1af-4bf5-9b21-c888cdc976e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:20.521213 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe31c84-d1af-4bf5-9b21-c888cdc976e9 service glance] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:20.531903 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1439/2929] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:19:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:19:30.396069 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:30.527175 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:19:53.492648 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35302812-2f89-413f-801d-3461657aa456 None None] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:53.585616 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35302812-2f89-413f-801d-3461657aa456 None None] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:53.590732 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:53.591603 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:53.591935 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:53.592359 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] PATH_INFO: `/v3/users/b46713e9dbf44c87b84c8ac819bea55e` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:53.604910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b46713e9dbf44c87b84c8ac819bea55e)` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:53.604910 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:53.795907 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b46713e9dbf44c87b84c8ac819bea55e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:53.796680 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b46713e9dbf44c87b84c8ac819bea55e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:53.806598 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b46713e9dbf44c87b84c8ac819bea55e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:53.807378 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b46713e9dbf44c87b84c8ac819bea55e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:53.879957 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.notifications [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b46713e9dbf44c87b84c8ac819bea55e'} {{(pid=93212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:53.900354 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1491/2930] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Oct 4 12:19:53 2022] DELETE /identity/v3/users/b46713e9dbf44c87b84c8ac819bea55e => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:19:53.905662 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071ecfa-e885-4bcb-853d-b04e9f60de15 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:53.906324 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071ecfa-e885-4bcb-853d-b04e9f60de15 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:53.906460 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071ecfa-e885-4bcb-853d-b04e9f60de15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:53.931890 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-7071ecfa-e885-4bcb-853d-b04e9f60de15 None None] MFA Rules not processed for user `374ef1a08ea74e7cb0bf480e25c8cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:19:53.933190 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7071ecfa-e885-4bcb-853d-b04e9f60de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:54.006103 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1440/2931] 10.222.0.15 () {58 vars in 958 bytes} [Tue Oct 4 12:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:19:54.017917 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:54.020385 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:54.094358 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ee76b-01a0-4a55-9549-0be4f6128251 admin admin] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:54.099030 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7fc20-60ab-48a4-b9a4-6268e7ecec96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:54.099665 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7fc20-60ab-48a4-b9a4-6268e7ecec96 service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:54.099947 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7fc20-60ab-48a4-b9a4-6268e7ecec96 service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:54.100252 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7fc20-60ab-48a4-b9a4-6268e7ecec96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:54.112201 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7fc20-60ab-48a4-b9a4-6268e7ecec96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:54.115474 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7fc20-60ab-48a4-b9a4-6268e7ecec96 service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:54.134315 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1492/2932] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:19:54.660944 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071ecfa-e885-4bcb-853d-b04e9f60de15 None None] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:19:54.667891 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-7071ecfa-e885-4bcb-853d-b04e9f60de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:19:54.685196 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071ecfa-e885-4bcb-853d-b04e9f60de15 None None] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:19:54.693111 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '374ef1a08ea74e7cb0bf480e25c8cdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a155f451dd324796b4bda64d212b8b97', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:19:54.698325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:19:54.698325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:19:54.698325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] PATH_INFO: `/v3/projects/af1931b378b64fdebd222ed54491369c` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:19:54.698325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af1931b378b64fdebd222ed54491369c)` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:19:54.698325 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:19:54.833057 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af1931b378b64fdebd222ed54491369c', 'request_id': 'req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:54.840539 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af1931b378b64fdebd222ed54491369c', 'request_id': 'req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:54.848351 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af1931b378b64fdebd222ed54491369c', 'request_id': 'req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9'} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:54.936733 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] The token cache is being invalidate because project af1931b378b64fdebd222ed54491369c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:19:54.937494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:54.937764 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.notifications [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:19:54.939486 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1441/2933] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Oct 4 12:19:54 2022] DELETE /identity/v3/projects/af1931b378b64fdebd222ed54491369c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:20:04.125298 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:20:04.932771 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:20:45.720896 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7fc20-60ab-48a4-b9a4-6268e7ecec96 service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:20:45.722855 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-42f7fc20-60ab-48a4-b9a4-6268e7ecec96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:20:45.840012 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7fc20-60ab-48a4-b9a4-6268e7ecec96 service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:20:45.910101 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d8638-ad13-43a9-b2ae-955f99808e1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16374dbb6d084ddfa3b49afd19c7c325', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:20:45.910726 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d8638-ad13-43a9-b2ae-955f99808e1d service placement] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:20:45.911003 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d8638-ad13-43a9-b2ae-955f99808e1d service placement] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:20:45.911270 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d8638-ad13-43a9-b2ae-955f99808e1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:20:45.913149 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-784d8638-ad13-43a9-b2ae-955f99808e1d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:20:45.925661 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d8638-ad13-43a9-b2ae-955f99808e1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:20:45.927362 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d8638-ad13-43a9-b2ae-955f99808e1d service placement] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:20:45.939784 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1493/2934] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:20:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:20:55.933946 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=67 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:23:01.688494 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] Authenticating user token {{(pid=93213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:23:01.690842 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:23:01.813455 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d850a01-e1fb-4da9-8e9b-6635ed9c63a9 admin admin] Validating token access rules against request {{(pid=93213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:23:01.818005 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d6d08-5944-48bd-b52d-2b4c41075c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:23:01.818687 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d6d08-5944-48bd-b52d-2b4c41075c0e service neutron] REQUEST_METHOD: `GET` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:23:01.819153 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d6d08-5944-48bd-b52d-2b4c41075c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:23:01.819518 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d6d08-5944-48bd-b52d-2b4c41075c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:23:01.822260 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-c47d6d08-5944-48bd-b52d-2b4c41075c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:23:01.925538 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d6d08-5944-48bd-b52d-2b4c41075c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:23:01.927035 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d6d08-5944-48bd-b52d-2b4c41075c0e service neutron] RBAC: Authorization granted {{(pid=93213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:23:01.951261 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1442/2935] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:23:01.959242 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d8638-ad13-43a9-b2ae-955f99808e1d service placement] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:23:01.976860 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d8638-ad13-43a9-b2ae-955f99808e1d service placement] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:23:01.983746 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9cc4c-7a92-4064-b31f-57f6cea4c46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:23:01.984805 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9cc4c-7a92-4064-b31f-57f6cea4c46c service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:23:01.985303 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9cc4c-7a92-4064-b31f-57f6cea4c46c service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:23:01.985801 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9cc4c-7a92-4064-b31f-57f6cea4c46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:23:01.989074 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.fernet_utils [None req-ceb9cc4c-7a92-4064-b31f-57f6cea4c46c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application [None req-ceb9cc4c-7a92-4064-b31f-57f6cea4c46c service neutron] Could not find project: 037b4d6af496475da87a47a9fd8d438d.: keystone.exception.ProjectNotFound: Could not find project: 037b4d6af496475da87a47a9fd8d438d. Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 04 12:23:02.027683 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:23:02.029876 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:23:02.032466 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application with Lock( Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:23:02.034833 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:23:02.036820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 037b4d6af496475da87a47a9fd8d438d. Oct 04 12:23:02.036820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: ERROR keystone.server.flask.application Oct 04 12:23:02.036820 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1494/2936] 10.222.0.15 () {60 vars in 1396 bytes} [Tue Oct 4 12:23:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 113 bytes in 71 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 04 12:23:02.076384 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d59994-88c1-409e-ba3a-0a8789d961d5 None None] REQUEST_METHOD: `POST` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:23:02.076730 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d59994-88c1-409e-ba3a-0a8789d961d5 None None] SCRIPT_NAME: `/identity` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:23:02.077054 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d59994-88c1-409e-ba3a-0a8789d961d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:23:02.147909 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.auth.core [None req-68d59994-88c1-409e-ba3a-0a8789d961d5 None None] MFA Rules not processed for user `ed14c1bc105d4c5cade5813edc1f0dac`. Rule list: `[]` (Enabled: `True`). {{(pid=93213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:23:02.149395 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG keystone.common.fernet_utils [None req-68d59994-88c1-409e-ba3a-0a8789d961d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:23:02.159469 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: [pid: 93213|app: 0|req: 1443/2937] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Oct 4 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:23:02.170628 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9cc4c-7a92-4064-b31f-57f6cea4c46c service neutron] Authenticating user token {{(pid=93212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:23:02.181037 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9cc4c-7a92-4064-b31f-57f6cea4c46c service neutron] Validating token access rules against request {{(pid=93212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:23:02.185781 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f195017-5078-4a32-b37f-020cf0ebd6df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '073e43c049224af29e9f6d426ea47e84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db48c0e368b84afdba6bdfe3c37065f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:23:02.186438 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f195017-5078-4a32-b37f-020cf0ebd6df service neutron] REQUEST_METHOD: `GET` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:23:02.186870 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f195017-5078-4a32-b37f-020cf0ebd6df service neutron] SCRIPT_NAME: `/identity` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:23:02.187209 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f195017-5078-4a32-b37f-020cf0ebd6df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:23:02.199049 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f195017-5078-4a32-b37f-020cf0ebd6df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:23:02.200717 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f195017-5078-4a32-b37f-020cf0ebd6df service neutron] RBAC: Authorization granted {{(pid=93212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:23:02.213751 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: [pid: 93212|app: 0|req: 1495/2938] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Oct 4 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:23:12.144642 dv-cinder-860212-2-iscsi devstack@keystone.service[93213]: DEBUG dbcounter [-] [93213] Writing DB stats keystone:SELECT=92 {{(pid=93213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:23:12.207759 dv-cinder-860212-2-iscsi devstack@keystone.service[93212]: DEBUG dbcounter [-] [93212] Writing DB stats keystone:SELECT=10 {{(pid=93212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}